-- Logs begin at Mon 2022-07-25 17:50:37 UTC, end at Mon 2022-07-25 19:47:52 UTC. -- Jul 25 18:41:22.819156 dv-cinder-820027-16-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 25 18:41:22.831045 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 25 18:41:22 2022] *** Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: nodename: dv-cinder-820027-16-iscsi Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: machine: x86_64 Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: clock source: unix Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: pcre jit disabled Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: detected number of CPU cores: 4 Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: current working directory: / Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: detected binary path: /usr/bin/uwsgi-core Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: your processes number limit is 47680 Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: your memory page size is 4096 bytes Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: detected max file descriptor number: 2048 Jul 25 18:41:22.832800 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: lock engine: pthread robust mutexes Jul 25 18:41:22.834005 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: thunder lock: enabled Jul 25 18:41:22.834005 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 25 18:41:22.834005 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 25 18:41:22.873563 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: Python main interpreter initialized at 0x556dc63189c0 Jul 25 18:41:22.873563 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: python threads support enabled Jul 25 18:41:22.873563 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: your server socket listen backlog is limited to 100 connections Jul 25 18:41:22.873563 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: your mercy for graceful operations on workers is 90 seconds Jul 25 18:41:22.873946 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: mapped 403077 bytes (393 KB) for 2 cores Jul 25 18:41:22.874023 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: *** Operational MODE: preforking *** Jul 25 18:41:22.874276 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: *** uWSGI is running in multiple interpreter mode *** Jul 25 18:41:22.874276 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: spawned uWSGI master process (pid: 86891) Jul 25 18:41:22.874514 dv-cinder-820027-16-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 25 18:41:22.875610 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: spawned uWSGI worker 1 (pid: 86905, cores: 1) Jul 25 18:41:22.875610 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: spawned uWSGI worker 2 (pid: 86906, cores: 1) Jul 25 18:41:22.875610 dv-cinder-820027-16-iscsi devstack@keystone.service[86891]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 25 18:41:28.475400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 18:41:28.475400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: from cryptography.utils import int_from_bytes Jul 25 18:41:28.476231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 18:41:28.476231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: from cryptography.utils import int_from_bytes Jul 25 18:41:28.527242 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 18:41:28.527451 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 18:41:28.527654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] command line args: [] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 18:41:28.528014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 18:41:28.528198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ================================================================================ {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 18:41:28.528459 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] admin_token = **** {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.528711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] config_dir = [] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.528901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.529085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] config_source = [] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.529268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] debug = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.529468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.529966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.530049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.530266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.530556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.530792 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] list_limit = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.530975 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] log_config_append = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.531155 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.531362 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] log_dir = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.532033 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] log_file = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.532348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.532842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.533132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.533390 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.533579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.533762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.533988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.534240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.534504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.534743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.534986 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.535222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.535474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.535724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.536005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.536500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.536687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] publish_errors = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.536945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.537314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.537424 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.537678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.537918 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.538154 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] standard_threads = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.538409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.538658 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.538837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.539016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] use_journal = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.539194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] use_json = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.539372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] use_stderr = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.539549 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] use_syslog = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.539784 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.540045 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.540285 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.540548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.540791 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.541042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.541282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.541531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.541798 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.542076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.542341 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.542604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.542843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.543091 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.543331 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.543580 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.543829 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.544068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.544326 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.544582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.544830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.545069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.545324 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.545588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.545843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.546095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.546347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.546634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.546962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.547758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.548100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.548365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.548790 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.549029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.549282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.549536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.549777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.550031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.550269 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.550535 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.550775 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.551009 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.551243 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.551476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.551709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.551948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.552385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.552630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.552872 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.553107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.553346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.553585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.553839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.554073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.554313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.554574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.554661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 18:41:28.554661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: from cryptography.utils import int_from_bytes Jul 25 18:41:28.555169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.555242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 18:41:28.555242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: from cryptography.utils import int_from_bytes Jul 25 18:41:28.555377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.555377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.555512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.555728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.555992 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.556209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.556542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.556817 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.557060 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.557302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.557544 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.557784 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.558117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.558302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.558581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.558834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.559078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.559336 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.559557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.559808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.560049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.560290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.560537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.560778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.561019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.561258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.561497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.561736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.562001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.562245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.562511 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.562782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.563085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.563426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.563760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.564011 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.564284 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.564522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.564754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.565044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.565281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.565527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.565768 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.566022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.566266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.571704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.571704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.571704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.571704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] auth.external = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.571704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.571704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.571704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.571704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] auth.password = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.571704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] auth.token = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] credential.caching = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.572745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.573233 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.573290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.573526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.573767 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.574026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.574268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] federation.caching = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.574533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.574786 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.575039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.575363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.575522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.575763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.576073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.576314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.576573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.576813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.577065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.577309 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.caching = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.577560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.577812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.578053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.578296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.578636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.579065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.579441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.579703 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.579965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.580207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.580446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.580757 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.580936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.581189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.581444 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.581706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.581961 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.582386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.582598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.582841 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.583081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.583322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.583557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.583797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.584043 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.584288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.584542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.584777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.585032 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.585282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.585520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.604515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 18:41:28.604748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 18:41:28.604887 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] command line args: [] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 18:41:28.605311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 18:41:28.605409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ================================================================================ {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 18:41:28.605604 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] admin_token = **** {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.605862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] config_dir = [] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.606056 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.606246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] config_source = [] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.606463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] debug = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.606694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.606980 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.607127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.607389 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.607646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.607887 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] list_limit = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.608074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] log_config_append = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.608257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.608439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] log_dir = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.608622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] log_file = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.608867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.609126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.609382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.609700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.610453 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.610712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.610976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.611233 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.611475 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.611723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.611960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.612203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.612460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.612716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.613004 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.613542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.613728 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] publish_errors = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.613970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.614209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.617467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.617467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.617467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] standard_threads = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.617467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.617467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.617467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.617467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] use_journal = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.617467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] use_json = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.617467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] use_stderr = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.618063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] use_syslog = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.618063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 18:41:28.618063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.618063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.618063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.618063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.618063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.618472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.618533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.618781 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.619040 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.619295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.619555 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.619800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.620055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.620373 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.620555 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.620815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.621057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.621321 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.621594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.621845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.622131 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.622425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.622703 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.622965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.623235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.623463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.623702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.623946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.624184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.624454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.624712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.624953 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.625191 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.625511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.625816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.626014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.626268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.626605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.626924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.627175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.627435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.627657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.627898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.628137 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.628388 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.628642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.628973 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.629127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.629366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.629610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.629854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.630111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.630370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.630624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.630868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.631111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.631355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.631604 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.631840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.632100 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.632344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.632589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.632829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.633100 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.633344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.633582 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.633828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.634072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.634382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.634675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.634894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.635155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.635400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.635643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.635885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.636145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.636396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.636643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.636894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.637141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.637387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.637631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.637875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.638119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.638423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.638680 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.638934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.639182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.639435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.639680 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.639938 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.640183 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.640424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.640665 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.640976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.641379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.641734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.642061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.642415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.642689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.642938 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.643207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.643484 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.643738 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.643981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] auth.external = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.644222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.644531 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.644774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.645014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] auth.password = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.645254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] auth.token = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.645686 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.645813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.646046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.646292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.646578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.646836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.647077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.647321 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] credential.caching = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.647578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.647836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.648094 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.648341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.648593 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.648836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.649114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.649411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.649625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.649887 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.650186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.650468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.650741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.650993 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.651259 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.651507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] federation.caching = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.651754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.652011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.652266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.652504 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.652760 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.653006 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.653270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.653513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.653837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.654024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.654280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.654554 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.caching = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.654811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.655091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.655315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.655559 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.655814 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.656055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.656421 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.657450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.657918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.658063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.658305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.658575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.658834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.659094 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.659410 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.659618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.659874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.660158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.660407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.660653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.660894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.661154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.661397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.661647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.661899 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.662150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.662454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.662678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.662936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.663191 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.663435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.663690 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.663945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.664184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.664428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.664665 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.664912 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.665263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.665523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.665772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.666020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.666278 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.666627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.666804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.667042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.667299 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.667555 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.667833 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.668080 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.668323 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.668560 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.668810 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.669078 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.669316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.669572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.669827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.670080 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.670342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.670585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.670830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.671071 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.671316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.671559 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.671816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.672905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.673163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.673320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.673443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.673644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.673773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.673995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.674116 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.674369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.674491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.674605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.674731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.674848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.674963 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.675102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.675220 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.675333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.675453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.675590 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.675708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.675848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.675951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.676092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.676222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.676357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.676545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.676663 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.676775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.676888 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.677009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.677131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.677245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.677364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.677482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.677603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.677710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.677823 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.677937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.678050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.678164 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.678276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.678421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.678539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.678652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.678769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] resource.caching = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.678883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.678995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.679116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.679232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.679344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.679445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.679523 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.679695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.679859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.679979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.680093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.680208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.680320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.680433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.680560 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.680662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.680779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.680893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.681005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.681118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.681238 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] role.caching = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.681353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.681493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] role.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.681610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.681723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.681836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.681983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.682101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.682240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.682397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.682514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.682627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.682774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.682894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.683073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.683216 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.683332 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.683447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.683569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.683684 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.683797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.683910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.684022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.684143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.684257 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.684369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.684483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.684596 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.684710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.684830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.684945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.685057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.685266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.685434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.685548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.685680 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.685785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.685940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.686045 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.686198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.686312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.691784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.691930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692082 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692153 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] resource.caching = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692518 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.692881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.693065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.693143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.693216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.693332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.693408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.693479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.693479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.693663 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.693740 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.693811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.693881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] token.caching = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.693952 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.694022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.694101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] role.caching = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.694173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.694243 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] role.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.694319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.694411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.694483 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.694566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.694639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.694704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.694774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.694886 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.694980 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.695063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.695136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.695207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.695278 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.695349 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.695420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.695644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.695723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.695794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.695865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.695936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.696007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.696086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.696158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.696275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.696350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.696421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.696500 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.696573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.696644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.696714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.696785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.connection = **** {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.696865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.696988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697345 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697691 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697832 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.697979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.698052 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.698123 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.698193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.698264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.698395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.698485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.698559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.698629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.698700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.698777 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.698857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.698930 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.699001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.699072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.699142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.699213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.699394 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.699482 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.699633 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.699713 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.699878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] token.caching = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.699956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86906) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 18:41:28.700123 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.700381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.700621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.700886 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.701143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.701391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.701643 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.701894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.702310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.702425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.702721 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.702965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.703267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.703529 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.704283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.704538 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.704827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.705096 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.connection = **** {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.705350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.705609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.705851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.706101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.706368 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.706613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.706857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.707102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.707353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.707597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.707855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.708098 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.708355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.708664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.708853 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.709132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.709382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.709626 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 18:41:28.709822 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86905) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 18:41:28.760095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.760095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.760095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.760498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.760498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.760498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.760498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.769589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.788796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.789028 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.789587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.789822 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.843245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.845468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.894781 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.898398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.898644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.979850 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.980074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.980299 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.980475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.980566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.980713 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.980798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.980926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.981008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.981151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.981374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:28.981593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.005355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.005581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.005813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.005908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.006015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.006098 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.006231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.006357 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.006468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.006552 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.006683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.006898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.006898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.007188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.018731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.019055 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.034862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.035035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.035203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.035297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.035376 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.035452 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.035526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.035641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.035728 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.035867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.035950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.036083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86906) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.036175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.036302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86905) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 18:41:29.053540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.053757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 18:41:29.054100 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.054208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.054315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 18:41:29.054421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 18:41:29.054591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.054679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.054856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.054983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 18:41:29.055100 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.055334 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.055472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.055725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.055956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.056219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.066510 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.066802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.067072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.067357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.067822 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.068190 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.068386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.068673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.075683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.077004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.079492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.079963 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.080245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.080524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.081066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.081166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.081331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.081420 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.081604 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.081743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.081874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.081956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.082146 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.082225 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.082432 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.082515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.082706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.082983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.083255 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.083528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.083797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.111730 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.111889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.113359 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.113547 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.116521 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.116615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.119044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.119250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.119344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.119540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.119649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.119816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.127019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.127306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.127537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.127827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.131859 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.132142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.132425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.132515 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.132710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.132797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.132983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.133073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.133256 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.133355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.133524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.133618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.133790 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.133878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.134066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.134350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.153101 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.153372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.153895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.154250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.158920 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.160149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.162701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.162999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.163815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.163939 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.168001 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.169139 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.173913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.174076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.175499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.175803 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.176085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.176363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.176642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.176924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.177021 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.177251 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.177545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.177832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.178113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.178426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.196525 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.196817 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.197085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.197348 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.200194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.200490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.200762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.201033 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.206003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.207890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.208163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.209634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.211232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.211386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.211511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.211662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.211793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.212079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.212357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.214746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.215031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.215649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.215776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.215997 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.226425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.226749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.226997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.227280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.227562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.227849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.228125 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.229829 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.230247 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.230427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.230717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.231001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.231284 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.231562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.246083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.249362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.250039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.253370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.254229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.255013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.256658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.256932 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.257212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.257562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.259231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.261032 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.261315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.261598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.264474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.266381 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.270295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.270591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.270821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.270909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.271110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.273596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.277015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.277307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.277593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.277868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.280666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.280967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.281248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.287681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.287977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.288261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.289367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86906) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.289652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.289940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.290218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.290527 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.290787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.291059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.291335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.291620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.291893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.292171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.292433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.292710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.292970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86906) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.296469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86905) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 18:41:29.296857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.297091 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.297409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.297689 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.297957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.298231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.298532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.298803 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.299203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.299480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.299746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.300029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.300291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86905) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 18:41:29.331673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x556dc63189c0 pid: 86906 (default app) Jul 25 18:41:29.335245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f280e8-a2bb-4626-bec6-af63cceea37f None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:29.335541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f280e8-a2bb-4626-bec6-af63cceea37f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:29.335819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f280e8-a2bb-4626-bec6-af63cceea37f None None] PATH_INFO: `/v3` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:29.340078 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Mon Jul 25 18:41:29 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:41:29.340332 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x556dc63189c0 pid: 86905 (default app) Jul 25 18:41:38.598540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84eb37a-73be-496d-943a-f361219f777e None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:38.599095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84eb37a-73be-496d-943a-f361219f777e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:38.599182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84eb37a-73be-496d-943a-f361219f777e None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:38.600779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 2/2] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:38.609304 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5641a-7f6f-496c-957f-ad2d80ca02e3 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:38.609776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5641a-7f6f-496c-957f-ad2d80ca02e3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:38.609860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5641a-7f6f-496c-957f-ad2d80ca02e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:38.683078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: INFO dbcounter [None req-49e5641a-7f6f-496c-957f-ad2d80ca02e3 None None] Registered counter for database keystone Jul 25 18:41:38.705621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG oslo_db.sqlalchemy.engines [None req-49e5641a-7f6f-496c-957f-ad2d80ca02e3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86905) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 18:41:38.712163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writer thread running {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 18:41:38.892850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG passlib.handlers.bcrypt [None req-49e5641a-7f6f-496c-957f-ad2d80ca02e3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86905) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 18:41:38.893148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG passlib.handlers.bcrypt [None req-49e5641a-7f6f-496c-957f-ad2d80ca02e3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86905) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 18:41:38.949346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-49e5641a-7f6f-496c-957f-ad2d80ca02e3 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:38.951100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-49e5641a-7f6f-496c-957f-ad2d80ca02e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:39.087736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1/3] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 18:41:39.233107 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84eb37a-73be-496d-943a-f361219f777e None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:39.303768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: INFO dbcounter [None req-d84eb37a-73be-496d-943a-f361219f777e None None] Registered counter for database keystone Jul 25 18:41:39.324388 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG oslo_db.sqlalchemy.engines [None req-d84eb37a-73be-496d-943a-f361219f777e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86906) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 18:41:39.330880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writer thread running {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 18:41:39.453337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84eb37a-73be-496d-943a-f361219f777e None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:39.457927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff77fee-9a5a-43d6-bc18-94a691e8d0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:39.458639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff77fee-9a5a-43d6-bc18-94a691e8d0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:39.458933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff77fee-9a5a-43d6-bc18-94a691e8d0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:39.459209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff77fee-9a5a-43d6-bc18-94a691e8d0c8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:39.468062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff77fee-9a5a-43d6-bc18-94a691e8d0c8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:39.520436 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5ff77fee-9a5a-43d6-bc18-94a691e8d0c8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 18:41:39.520779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff77fee-9a5a-43d6-bc18-94a691e8d0c8 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:39.531395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 3/4] 10.222.0.17 () {58 vars in 1324 bytes} [Mon Jul 25 18:41:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 18:41:39.537649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5641a-7f6f-496c-957f-ad2d80ca02e3 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:39.552934 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5641a-7f6f-496c-957f-ad2d80ca02e3 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:39.557459 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2252b0c-1e5e-4fd6-9648-ef4ab86a40c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:39.558663 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252b0c-1e5e-4fd6-9648-ef4ab86a40c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:39.558923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252b0c-1e5e-4fd6-9648-ef4ab86a40c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:39.559204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252b0c-1e5e-4fd6-9648-ef4ab86a40c8 admin admin] PATH_INFO: `/v3/projects/c39adca8a83c49e4a2ab45e68b3a9724` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:39.560855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2252b0c-1e5e-4fd6-9648-ef4ab86a40c8 admin admin] RBAC: Authorizing `identity:get_project(project_id=c39adca8a83c49e4a2ab45e68b3a9724)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:39.612695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b2252b0c-1e5e-4fd6-9648-ef4ab86a40c8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 18:41:39.613304 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2252b0c-1e5e-4fd6-9648-ef4ab86a40c8 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:39.617919 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 2/5] 10.222.0.17 () {56 vars in 1207 bytes} [Mon Jul 25 18:41:39 2022] GET /identity/v3/projects/c39adca8a83c49e4a2ab45e68b3a9724 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:41:41.629308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250c62-1614-4658-a458-b0e1fff71e1d None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:41.629834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250c62-1614-4658-a458-b0e1fff71e1d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:41.629949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250c62-1614-4658-a458-b0e1fff71e1d None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:41.634041 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 4/6] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:41.639791 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fc181-6a3d-478b-a3a9-e6aadf9020df None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:41.640250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fc181-6a3d-478b-a3a9-e6aadf9020df None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:41.640364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fc181-6a3d-478b-a3a9-e6aadf9020df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:41.662421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-5c3fc181-6a3d-478b-a3a9-e6aadf9020df None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:41.663977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5c3fc181-6a3d-478b-a3a9-e6aadf9020df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:41.672216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 3/7] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 25 18:41:41.696236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e250c62-1614-4658-a458-b0e1fff71e1d None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:41.708632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e250c62-1614-4658-a458-b0e1fff71e1d None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:41.712894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3e467-202b-4310-9196-a952f178e049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:41.713536 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3e467-202b-4310-9196-a952f178e049 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:41.713825 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3e467-202b-4310-9196-a952f178e049 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:41.714106 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3e467-202b-4310-9196-a952f178e049 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:41.722534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3e467-202b-4310-9196-a952f178e049 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:41.724323 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3e467-202b-4310-9196-a952f178e049 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:41.733254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 5/8] 10.222.0.17 () {58 vars in 1324 bytes} [Mon Jul 25 18:41:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 18:41:41.739715 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3fc181-6a3d-478b-a3a9-e6aadf9020df None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:41.749489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3fc181-6a3d-478b-a3a9-e6aadf9020df None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:41.754735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c1f6d-f02c-4045-aaa1-3113ae18175e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:41.755624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c1f6d-f02c-4045-aaa1-3113ae18175e admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:41.755918 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c1f6d-f02c-4045-aaa1-3113ae18175e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:41.756201 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c1f6d-f02c-4045-aaa1-3113ae18175e admin admin] PATH_INFO: `/v3/users/4a0aec528900484294cd86eb90b642f7` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:41.757666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c1f6d-f02c-4045-aaa1-3113ae18175e admin admin] RBAC: Authorizing `identity:get_user(user_id=4a0aec528900484294cd86eb90b642f7)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:41.759660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c1f6d-f02c-4045-aaa1-3113ae18175e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:41.761406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 4/9] 10.222.0.17 () {56 vars in 1198 bytes} [Mon Jul 25 18:41:41 2022] GET /identity/v3/users/4a0aec528900484294cd86eb90b642f7 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:41:43.857513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be510d3-65d5-4892-a7e7-55991efcf677 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:43.858058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be510d3-65d5-4892-a7e7-55991efcf677 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:43.858058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be510d3-65d5-4892-a7e7-55991efcf677 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:43.859643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 6/10] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:43.865729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df545d6-4e26-4434-a6ea-226fbf1e80e3 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:43.866169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df545d6-4e26-4434-a6ea-226fbf1e80e3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:43.866305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df545d6-4e26-4434-a6ea-226fbf1e80e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:43.885648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-6df545d6-4e26-4434-a6ea-226fbf1e80e3 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:43.886894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6df545d6-4e26-4434-a6ea-226fbf1e80e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:43.956561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708bcd1-d9bc-4b6f-9cbe-0b28060ae6f6 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:43.956856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708bcd1-d9bc-4b6f-9cbe-0b28060ae6f6 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:43.957149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708bcd1-d9bc-4b6f-9cbe-0b28060ae6f6 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:43.958646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 7/11] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:41:43.964346 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadd786-3491-4ded-bfa0-fad8ee560917 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:43.964653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadd786-3491-4ded-bfa0-fad8ee560917 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:43.964943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadd786-3491-4ded-bfa0-fad8ee560917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:43.973895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 5/12] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:41:43.996708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df545d6-4e26-4434-a6ea-226fbf1e80e3 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:44.010582 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG passlib.handlers.bcrypt [None req-baadd786-3491-4ded-bfa0-fad8ee560917 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86906) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 18:41:44.010930 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG passlib.handlers.bcrypt [None req-baadd786-3491-4ded-bfa0-fad8ee560917 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86906) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 18:41:44.022486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:44.057661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-baadd786-3491-4ded-bfa0-fad8ee560917 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:44.059418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-baadd786-3491-4ded-bfa0-fad8ee560917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:44.106752 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df545d6-4e26-4434-a6ea-226fbf1e80e3 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:44.146400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff42ee-0236-4ed4-bb0b-869961e25f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:44.146986 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff42ee-0236-4ed4-bb0b-869961e25f1c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:44.147310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff42ee-0236-4ed4-bb0b-869961e25f1c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:44.147538 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff42ee-0236-4ed4-bb0b-869961e25f1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:44.162079 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff42ee-0236-4ed4-bb0b-869961e25f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:41:44.169210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 8/13] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-e7ff42ee-0236-4ed4-bb0b-869961e25f1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:41:44.173739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:41:44.174717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:41:44.175575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:41:44.175575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:41:44.175575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:41:44.175575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:41:44.175575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:41:44.175575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:41:44.175575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:41:44.175575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:41:44.175575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:41:44.175575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:41:44.175575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:41:44.175575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:41:44.175575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 6/14] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:41:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:41:44.181011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadd786-3491-4ded-bfa0-fad8ee560917 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:44.192711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff42ee-0236-4ed4-bb0b-869961e25f1c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:44.292666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:44.293440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadd786-3491-4ded-bfa0-fad8ee560917 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:44.295977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:44.311418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff42ee-0236-4ed4-bb0b-869961e25f1c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:44.337284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb0cd5-6e95-4d06-929c-5eb6ca78988e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:44.338149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb0cd5-6e95-4d06-929c-5eb6ca78988e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:44.338471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb0cd5-6e95-4d06-929c-5eb6ca78988e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:44.338755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb0cd5-6e95-4d06-929c-5eb6ca78988e None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:44.339705 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb0cd5-6e95-4d06-929c-5eb6ca78988e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:41:44.339998 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb0cd5-6e95-4d06-929c-5eb6ca78988e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:44.341449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb0cd5-6e95-4d06-929c-5eb6ca78988e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:44.351446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7bd15-0508-47d3-bc58-508a4a61551e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:44.351680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7bd15-0508-47d3-bc58-508a4a61551e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:44.351974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7bd15-0508-47d3-bc58-508a4a61551e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:44.352259 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7bd15-0508-47d3-bc58-508a4a61551e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:44.359322 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 9/15] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:41:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:41:44.360710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb0cd5-6e95-4d06-929c-5eb6ca78988e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:44.427763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7bd15-0508-47d3-bc58-508a4a61551e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:44.429183 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7bd15-0508-47d3-bc58-508a4a61551e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:44.468581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb0cd5-6e95-4d06-929c-5eb6ca78988e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:44.505873 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4248fd-c893-4da6-a606-48199850fc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:44.506797 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4248fd-c893-4da6-a606-48199850fc02 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:44.507086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4248fd-c893-4da6-a606-48199850fc02 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:44.507371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4248fd-c893-4da6-a606-48199850fc02 None admin] PATH_INFO: `/v3/users/4a0aec528900484294cd86eb90b642f7` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:44.508946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4248fd-c893-4da6-a606-48199850fc02 None admin] RBAC: Authorizing `identity:get_user(user_id=4a0aec528900484294cd86eb90b642f7)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:44.510449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4248fd-c893-4da6-a606-48199850fc02 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:44.512099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 10/16] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:41:44 2022] GET /identity/v3/users/4a0aec528900484294cd86eb90b642f7 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:41:44.513921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:44.518302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4248fd-c893-4da6-a606-48199850fc02 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:44.534240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 7/17] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:41:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 18:41:44.540147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7bd15-0508-47d3-bc58-508a4a61551e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:44.554206 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:44.631617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4248fd-c893-4da6-a606-48199850fc02 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:44.648068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7bd15-0508-47d3-bc58-508a4a61551e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:44.669948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341eae2-3852-40ef-a2c3-852510da5aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:44.670798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341eae2-3852-40ef-a2c3-852510da5aef None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:44.671099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341eae2-3852-40ef-a2c3-852510da5aef None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:44.671372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341eae2-3852-40ef-a2c3-852510da5aef None admin] PATH_INFO: `/v3/domains/default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:44.672606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341eae2-3852-40ef-a2c3-852510da5aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:44.674283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341eae2-3852-40ef-a2c3-852510da5aef None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:44.676195 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 11/18] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:41:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:41:44.682021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341eae2-3852-40ef-a2c3-852510da5aef None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:44.682966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76636861-7151-4083-a616-2d5249f44421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:44.683788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636861-7151-4083-a616-2d5249f44421 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:44.684077 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636861-7151-4083-a616-2d5249f44421 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:44.684371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636861-7151-4083-a616-2d5249f44421 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:44.695547 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76636861-7151-4083-a616-2d5249f44421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:44.697070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76636861-7151-4083-a616-2d5249f44421 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-76636861-7151-4083-a616-2d5249f44421 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:41:44.716448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:41:44.721410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:41:44.722228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:41:44.722228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:41:44.722228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:41:44.722228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:41:44.722228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 8/19] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:41:44.724662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76636861-7151-4083-a616-2d5249f44421 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:44.759223 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:44.788020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341eae2-3852-40ef-a2c3-852510da5aef None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:44.789586 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:44.824744 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f48e7-691b-4ccd-a39d-937f8e0d672f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:44.825409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f48e7-691b-4ccd-a39d-937f8e0d672f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:44.825700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f48e7-691b-4ccd-a39d-937f8e0d672f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:44.825825 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76636861-7151-4083-a616-2d5249f44421 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:44.825979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f48e7-691b-4ccd-a39d-937f8e0d672f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:44.826847 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f48e7-691b-4ccd-a39d-937f8e0d672f None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.domain.id=default, user.id=4a0aec528900484294cd86eb90b642f7) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:41:44.827163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f48e7-691b-4ccd-a39d-937f8e0d672f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:44.828641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f48e7-691b-4ccd-a39d-937f8e0d672f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:44.838036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 12/20] 10.222.0.17 () {56 vars in 1322 bytes} [Mon Jul 25 18:41:44 2022] GET /identity/v3/role_assignments?user.id=4a0aec528900484294cd86eb90b642f7&scope.domain.id=default&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:41:44.865668 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbbb1c-183d-4c69-b258-b623d183e5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:44.866378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbbb1c-183d-4c69-b258-b623d183e5e5 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:44.866917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbbb1c-183d-4c69-b258-b623d183e5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:44.866917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbbb1c-183d-4c69-b258-b623d183e5e5 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:44.867681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbbb1c-183d-4c69-b258-b623d183e5e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:41:44.867994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbbb1c-183d-4c69-b258-b623d183e5e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:44.869566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbbb1c-183d-4c69-b258-b623d183e5e5 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:44.889591 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 9/21] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:41:47.315571 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e4473-4b92-4643-9322-53f76976de65 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:47.317150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e4473-4b92-4643-9322-53f76976de65 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:47.317919 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e4473-4b92-4643-9322-53f76976de65 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:47.320747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 13/22] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:47.327089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc8300-2f32-4ef5-9a2d-37fc7e410d20 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:47.327929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc8300-2f32-4ef5-9a2d-37fc7e410d20 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:47.346840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc8300-2f32-4ef5-9a2d-37fc7e410d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:47.352682 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-c8bc8300-2f32-4ef5-9a2d-37fc7e410d20 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:47.354199 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c8bc8300-2f32-4ef5-9a2d-37fc7e410d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:47.438503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba8765-41d8-4847-ab27-96b3c2ea1108 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:47.439032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba8765-41d8-4847-ab27-96b3c2ea1108 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:47.440596 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba8765-41d8-4847-ab27-96b3c2ea1108 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:47.442296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 14/23] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:41:47.448912 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b47cb-c458-4a2a-a1da-3c1be3da085e None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:47.450093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b47cb-c458-4a2a-a1da-3c1be3da085e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:47.450621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b47cb-c458-4a2a-a1da-3c1be3da085e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:47.464750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 10/24] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:41:47.491552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-996b47cb-c458-4a2a-a1da-3c1be3da085e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:47.492809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-996b47cb-c458-4a2a-a1da-3c1be3da085e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:47.552929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc8300-2f32-4ef5-9a2d-37fc7e410d20 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:47.611131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:47.625338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 15/25] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:41:47.686455 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b47cb-c458-4a2a-a1da-3c1be3da085e None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:47.768867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc8300-2f32-4ef5-9a2d-37fc7e410d20 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:47.808357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:47.816658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b47cb-c458-4a2a-a1da-3c1be3da085e None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:47.857483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a9506-80b1-4cdb-b533-da216fa5efef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:47.858383 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a9506-80b1-4cdb-b533-da216fa5efef None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:47.858667 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a9506-80b1-4cdb-b533-da216fa5efef None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:47.859001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a9506-80b1-4cdb-b533-da216fa5efef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:47.863694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d6d70-c4e2-48f8-9b48-6f0894452f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:47.864799 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d6d70-c4e2-48f8-9b48-6f0894452f09 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:47.865318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d6d70-c4e2-48f8-9b48-6f0894452f09 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:47.865794 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d6d70-c4e2-48f8-9b48-6f0894452f09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:47.872078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a9506-80b1-4cdb-b533-da216fa5efef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-e02a9506-80b1-4cdb-b533-da216fa5efef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:41:47.881446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:41:47.882353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:41:47.883187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:41:47.883187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:41:47.883187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:41:47.883187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:41:47.883187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:41:47.883187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:41:47.883187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:41:47.883187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:41:47.883187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:41:47.883187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:41:47.883187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:41:47.883187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:41:47.886163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 11/26] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:41:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:41:47.888070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b67f94-76db-4ca2-9b19-a60621ca8b45 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:47.888586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b67f94-76db-4ca2-9b19-a60621ca8b45 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:47.889066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b67f94-76db-4ca2-9b19-a60621ca8b45 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:47.891048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 12/27] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:47.894685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b67f94-76db-4ca2-9b19-a60621ca8b45 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:47.913850 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d6d70-c4e2-48f8-9b48-6f0894452f09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:47.917091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d6d70-c4e2-48f8-9b48-6f0894452f09 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-bd6d6d70-c4e2-48f8-9b48-6f0894452f09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:41:47.948034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:41:47.948882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:41:47.949711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:41:47.949711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:41:47.949711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:41:47.949711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:41:47.952009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 16/28] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:41:47.953787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30dad0-a01c-47ac-88ab-fdbb47699a93 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:47.954302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30dad0-a01c-47ac-88ab-fdbb47699a93 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:47.954816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30dad0-a01c-47ac-88ab-fdbb47699a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:48.016133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-7c30dad0-a01c-47ac-88ab-fdbb47699a93 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:48.018768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-7c30dad0-a01c-47ac-88ab-fdbb47699a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:48.126884 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:48.151002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b67f94-76db-4ca2-9b19-a60621ca8b45 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:48.161614 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 17/29] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:41:48.162809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30dad0-a01c-47ac-88ab-fdbb47699a93 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:48.214958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62cc98-bd54-436a-96f3-164c127e09c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:48.216266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62cc98-bd54-436a-96f3-164c127e09c3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:48.216964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62cc98-bd54-436a-96f3-164c127e09c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:48.217667 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62cc98-bd54-436a-96f3-164c127e09c3 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:48.219301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62cc98-bd54-436a-96f3-164c127e09c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:41:48.220350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62cc98-bd54-436a-96f3-164c127e09c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:48.223281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62cc98-bd54-436a-96f3-164c127e09c3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:48.245111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 13/30] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:41:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:41:48.255556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259733cd-6649-40a4-b08a-e6596832f3b5 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:48.256372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259733cd-6649-40a4-b08a-e6596832f3b5 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:48.257068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259733cd-6649-40a4-b08a-e6596832f3b5 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:48.259712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 14/31] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:48 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:48.261194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259733cd-6649-40a4-b08a-e6596832f3b5 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:48.266186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:48.297809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30dad0-a01c-47ac-88ab-fdbb47699a93 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:48.363326 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ebe04-ab5c-4054-8a2c-8e79a2bffc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:48.364755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ebe04-ab5c-4054-8a2c-8e79a2bffc12 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:48.365244 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ebe04-ab5c-4054-8a2c-8e79a2bffc12 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:48.365678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ebe04-ab5c-4054-8a2c-8e79a2bffc12 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:48.366646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ebe04-ab5c-4054-8a2c-8e79a2bffc12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:41:48.367104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ebe04-ab5c-4054-8a2c-8e79a2bffc12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:48.368781 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ebe04-ab5c-4054-8a2c-8e79a2bffc12 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:48.396025 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 18/32] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:41:48.396025 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ebe04-ab5c-4054-8a2c-8e79a2bffc12 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:48.446556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259733cd-6649-40a4-b08a-e6596832f3b5 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:48.490228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad393b7-4f0f-4bbe-b945-bb24172e1f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:48.491125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad393b7-4f0f-4bbe-b945-bb24172e1f3c None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:48.491420 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad393b7-4f0f-4bbe-b945-bb24172e1f3c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:48.491704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad393b7-4f0f-4bbe-b945-bb24172e1f3c None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:48.493339 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad393b7-4f0f-4bbe-b945-bb24172e1f3c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:48.494880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad393b7-4f0f-4bbe-b945-bb24172e1f3c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:48.527633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ebe04-ab5c-4054-8a2c-8e79a2bffc12 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:48.678994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb81c4-2afb-4b32-b82f-0f6e96a41a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:48.680166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb81c4-2afb-4b32-b82f-0f6e96a41a23 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:48.680635 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb81c4-2afb-4b32-b82f-0f6e96a41a23 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:48.681063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb81c4-2afb-4b32-b82f-0f6e96a41a23 None admin] PATH_INFO: `/v3/users/4a0aec528900484294cd86eb90b642f7` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:48.683315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb81c4-2afb-4b32-b82f-0f6e96a41a23 None admin] RBAC: Authorizing `identity:get_user(user_id=4a0aec528900484294cd86eb90b642f7)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:48.684996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb81c4-2afb-4b32-b82f-0f6e96a41a23 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:48.687416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 19/33] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:41:48 2022] GET /identity/v3/users/4a0aec528900484294cd86eb90b642f7 => generated 247 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:41:48.689155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5a020-4193-4134-8591-afb5a79ce610 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:48.690275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5a020-4193-4134-8591-afb5a79ce610 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:48.690897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5a020-4193-4134-8591-afb5a79ce610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:48.706557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:48.713696 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-1ed5a020-4193-4134-8591-afb5a79ce610 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:48.715445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1ed5a020-4193-4134-8591-afb5a79ce610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:48.802478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 20/34] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:41:48.803753 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5a020-4193-4134-8591-afb5a79ce610 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:48.856070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 15/35] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Jul 25 18:41:48 2022] POST /identity/v3/roles => generated 219 bytes in 595 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:41:48.857131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad393b7-4f0f-4bbe-b945-bb24172e1f3c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:48.871286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:48.947860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5a020-4193-4134-8591-afb5a79ce610 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:48.998280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbaf018-ef43-46ba-841d-dcd8319fc1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:48.999017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaf018-ef43-46ba-841d-dcd8319fc1b6 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:48.999307 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaf018-ef43-46ba-841d-dcd8319fc1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:48.999592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaf018-ef43-46ba-841d-dcd8319fc1b6 None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:49.000379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbaf018-ef43-46ba-841d-dcd8319fc1b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:49.008396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbaf018-ef43-46ba-841d-dcd8319fc1b6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:49.030712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:49.065685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad393b7-4f0f-4bbe-b945-bb24172e1f3c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:49.109627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8a423-6abe-4535-ace1-9f41a13091e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:49.110543 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8a423-6abe-4535-ace1-9f41a13091e9 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:49.110842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8a423-6abe-4535-ace1-9f41a13091e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:49.111132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8a423-6abe-4535-ace1-9f41a13091e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:49.113303 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8a423-6abe-4535-ace1-9f41a13091e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:49.115003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8a423-6abe-4535-ace1-9f41a13091e9 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:49.116959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 16/36] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:41:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:41:49.117516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8a423-6abe-4535-ace1-9f41a13091e9 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:49.220579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:49.246051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8a423-6abe-4535-ace1-9f41a13091e9 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:49.286041 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70e670-1e31-40fd-969a-4e9308718e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:49.287085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70e670-1e31-40fd-969a-4e9308718e96 None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:49.287688 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70e670-1e31-40fd-969a-4e9308718e96 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:49.288351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70e670-1e31-40fd-969a-4e9308718e96 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:49.289659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70e670-1e31-40fd-969a-4e9308718e96 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:49.291384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70e670-1e31-40fd-969a-4e9308718e96 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:49.368892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 21/37] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:41:48 2022] POST /identity/v3/projects => generated 301 bytes in 565 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:41:49.369697 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbaf018-ef43-46ba-841d-dcd8319fc1b6 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:49.495759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbaf018-ef43-46ba-841d-dcd8319fc1b6 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:49.502607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 17/38] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Jul 25 18:41:49 2022] POST /identity/v3/roles => generated 225 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:41:49.502607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3839699-e278-4fd7-8123-18aa92172298 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:49.502607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3839699-e278-4fd7-8123-18aa92172298 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:49.502607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3839699-e278-4fd7-8123-18aa92172298 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:49.502607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 18/39] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:49.502607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7366cb8-2ccd-4289-8f84-ff96194ff31c None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:49.502607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7366cb8-2ccd-4289-8f84-ff96194ff31c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:49.503681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7366cb8-2ccd-4289-8f84-ff96194ff31c None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:49.504086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 19/40] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:49.507374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117fe68-1e4f-4204-b39a-d4894b131847 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:49.507374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117fe68-1e4f-4204-b39a-d4894b131847 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:49.507627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117fe68-1e4f-4204-b39a-d4894b131847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:49.540722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-f117fe68-1e4f-4204-b39a-d4894b131847 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:49.543115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f117fe68-1e4f-4204-b39a-d4894b131847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:49.560943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b50b73-11d9-4af6-9740-125e06587c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:49.562020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b50b73-11d9-4af6-9740-125e06587c21 None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:49.562383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b50b73-11d9-4af6-9740-125e06587c21 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:49.562683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b50b73-11d9-4af6-9740-125e06587c21 None admin] PATH_INFO: `/v3/domains/default/users/4a0aec528900484294cd86eb90b642f7/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:49.568731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b50b73-11d9-4af6-9740-125e06587c21 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4a0aec528900484294cd86eb90b642f7, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:49.570757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b50b73-11d9-4af6-9740-125e06587c21 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:49.651761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 20/41] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:41:49.653050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aa59f-9892-42d6-8b4c-3abc996a8e1a None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:49.653347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aa59f-9892-42d6-8b4c-3abc996a8e1a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:49.653627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aa59f-9892-42d6-8b4c-3abc996a8e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:49.676695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-e51aa59f-9892-42d6-8b4c-3abc996a8e1a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:49.677943 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e51aa59f-9892-42d6-8b4c-3abc996a8e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:49.745199 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:49.745700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 22/42] 10.222.0.17 () {58 vars in 1361 bytes} [Mon Jul 25 18:41:49 2022] PUT /identity/v3/domains/default/users/4a0aec528900484294cd86eb90b642f7/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:41:49.746767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b50b73-11d9-4af6-9740-125e06587c21 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:49.755674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 21/43] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:41:49.782793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aa59f-9892-42d6-8b4c-3abc996a8e1a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:49.819741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:49.880066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b50b73-11d9-4af6-9740-125e06587c21 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:49.899806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aa59f-9892-42d6-8b4c-3abc996a8e1a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:49.919481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716618a4-fb5c-46bc-8354-895cfda253f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:49.920173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716618a4-fb5c-46bc-8354-895cfda253f7 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:49.920469 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716618a4-fb5c-46bc-8354-895cfda253f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:49.920753 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716618a4-fb5c-46bc-8354-895cfda253f7 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:49.921516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716618a4-fb5c-46bc-8354-895cfda253f7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:49.922986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716618a4-fb5c-46bc-8354-895cfda253f7 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:49.940922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1866b-8ccf-4e63-820c-e4f581c9eef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:49.942054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1866b-8ccf-4e63-820c-e4f581c9eef3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:49.942054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1866b-8ccf-4e63-820c-e4f581c9eef3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:49.942273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1866b-8ccf-4e63-820c-e4f581c9eef3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:49.943536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1866b-8ccf-4e63-820c-e4f581c9eef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:49.945008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1866b-8ccf-4e63-820c-e4f581c9eef3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:49.946685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 22/44] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:41:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:41:49.952320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1866b-8ccf-4e63-820c-e4f581c9eef3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:50.043946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 23/45] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Jul 25 18:41:49 2022] POST /identity/v3/roles => generated 223 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:41:50.061292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1866b-8ccf-4e63-820c-e4f581c9eef3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:50.067042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:50.114875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21eced-52cc-4a13-8fb2-2532701bc543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:50.115602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21eced-52cc-4a13-8fb2-2532701bc543 None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:50.115901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21eced-52cc-4a13-8fb2-2532701bc543 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:50.116187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21eced-52cc-4a13-8fb2-2532701bc543 None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:50.116940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21eced-52cc-4a13-8fb2-2532701bc543 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:50.118526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21eced-52cc-4a13-8fb2-2532701bc543 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:50.232368 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 23/46] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:41:49 2022] POST /identity/v3/projects => generated 312 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:41:51.883794 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91ded5-856c-4e0f-824b-47d155817d04 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:51.884320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91ded5-856c-4e0f-824b-47d155817d04 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:51.884430 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91ded5-856c-4e0f-824b-47d155817d04 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:51.885747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 24/47] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:51.891060 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec8158-c59c-4959-b436-e9481a13ad4f None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:51.891494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec8158-c59c-4959-b436-e9481a13ad4f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:51.891651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec8158-c59c-4959-b436-e9481a13ad4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:51.911301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-a3ec8158-c59c-4959-b436-e9481a13ad4f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:51.912536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a3ec8158-c59c-4959-b436-e9481a13ad4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:51.982270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 24/48] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:41:52.006072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91ded5-856c-4e0f-824b-47d155817d04 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:52.120791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91ded5-856c-4e0f-824b-47d155817d04 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:52.131356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:52.160004 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb99ff-0376-48b1-90f8-b29d31b3d621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:52.160487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb99ff-0376-48b1-90f8-b29d31b3d621 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:52.160775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb99ff-0376-48b1-90f8-b29d31b3d621 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:52.161118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb99ff-0376-48b1-90f8-b29d31b3d621 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:52.173585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb99ff-0376-48b1-90f8-b29d31b3d621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-31bb99ff-0376-48b1-90f8-b29d31b3d621 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:41:52.183066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:41:52.184057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:41:52.184801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:41:52.184801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:41:52.184801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:41:52.184801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:41:52.184801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:41:52.184801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:41:52.184801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:41:52.184801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:41:52.184801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:41:52.184801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:41:52.184801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:41:52.184801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:41:52.184801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 25/49] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:41:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:41:52.190309 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec8158-c59c-4959-b436-e9481a13ad4f None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:52.239881 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:52.297157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec8158-c59c-4959-b436-e9481a13ad4f None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:52.321705 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7944dd4-3c8c-44c7-8099-6ff5ad7200af None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:52.322181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7944dd4-3c8c-44c7-8099-6ff5ad7200af None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:52.322697 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7944dd4-3c8c-44c7-8099-6ff5ad7200af None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:52.325098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 26/50] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:41:52.331434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90201f5d-756b-4462-8088-6f384df85c1e None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:52.331741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90201f5d-756b-4462-8088-6f384df85c1e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:52.332035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90201f5d-756b-4462-8088-6f384df85c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:52.336891 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6816f8-089c-48ed-9363-7397e7a19bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:52.337543 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6816f8-089c-48ed-9363-7397e7a19bba None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:52.337835 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6816f8-089c-48ed-9363-7397e7a19bba None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:52.338127 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6816f8-089c-48ed-9363-7397e7a19bba None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:52.338926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6816f8-089c-48ed-9363-7397e7a19bba None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:41:52.339372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6816f8-089c-48ed-9363-7397e7a19bba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:52.340662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6816f8-089c-48ed-9363-7397e7a19bba None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:52.352409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 25/51] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:41:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:41:52.353491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-90201f5d-756b-4462-8088-6f384df85c1e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:52.355171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-90201f5d-756b-4462-8088-6f384df85c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:52.359060 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6816f8-089c-48ed-9363-7397e7a19bba None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:52.430201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 27/52] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:41:52.454072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90201f5d-756b-4462-8088-6f384df85c1e None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:52.463308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6816f8-089c-48ed-9363-7397e7a19bba None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:52.471136 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:52.500314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24efb72-b62c-41d8-9af2-8ae6c102a3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:52.501218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24efb72-b62c-41d8-9af2-8ae6c102a3e6 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:52.501509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24efb72-b62c-41d8-9af2-8ae6c102a3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:52.501796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24efb72-b62c-41d8-9af2-8ae6c102a3e6 None admin] PATH_INFO: `/v3/users/4a0aec528900484294cd86eb90b642f7` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:52.503747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24efb72-b62c-41d8-9af2-8ae6c102a3e6 None admin] RBAC: Authorizing `identity:get_user(user_id=4a0aec528900484294cd86eb90b642f7)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:52.505239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24efb72-b62c-41d8-9af2-8ae6c102a3e6 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:52.507150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 26/53] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:41:52 2022] GET /identity/v3/users/4a0aec528900484294cd86eb90b642f7 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:41:52.512556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24efb72-b62c-41d8-9af2-8ae6c102a3e6 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:52.560910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:52.565745 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90201f5d-756b-4462-8088-6f384df85c1e None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:52.607347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb897f9-6d81-452e-806a-2c9399be8f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:52.608209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb897f9-6d81-452e-806a-2c9399be8f94 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:52.608491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb897f9-6d81-452e-806a-2c9399be8f94 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:52.608773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb897f9-6d81-452e-806a-2c9399be8f94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:52.610030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb897f9-6d81-452e-806a-2c9399be8f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:52.611535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb897f9-6d81-452e-806a-2c9399be8f94 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:52.613967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 28/54] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:41:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:41:52.619293 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24efb72-b62c-41d8-9af2-8ae6c102a3e6 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:52.619441 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb897f9-6d81-452e-806a-2c9399be8f94 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:52.659370 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae60bffd-743f-43c4-8702-3341db2e7ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:52.660196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60bffd-743f-43c4-8702-3341db2e7ac0 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:52.660480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60bffd-743f-43c4-8702-3341db2e7ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:52.660759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60bffd-743f-43c4-8702-3341db2e7ac0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:52.662083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae60bffd-743f-43c4-8702-3341db2e7ac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:52.663600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae60bffd-743f-43c4-8702-3341db2e7ac0 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:52.665252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 27/55] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:41:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:41:52.671153 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae60bffd-743f-43c4-8702-3341db2e7ac0 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:52.716248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:52.730399 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb897f9-6d81-452e-806a-2c9399be8f94 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:52.765955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b89700-0521-4617-8276-483bc4b22da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:52.766763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b89700-0521-4617-8276-483bc4b22da3 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:52.767057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b89700-0521-4617-8276-483bc4b22da3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:52.767801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b89700-0521-4617-8276-483bc4b22da3 None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:52.768129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b89700-0521-4617-8276-483bc4b22da3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:52.769596 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b89700-0521-4617-8276-483bc4b22da3 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:52.775196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae60bffd-743f-43c4-8702-3341db2e7ac0 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:52.811586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519aecf-e70a-4dc5-81dc-ebd0bd34aec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:52.812285 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519aecf-e70a-4dc5-81dc-ebd0bd34aec1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:52.812572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519aecf-e70a-4dc5-81dc-ebd0bd34aec1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:52.812847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519aecf-e70a-4dc5-81dc-ebd0bd34aec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:52.813678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519aecf-e70a-4dc5-81dc-ebd0bd34aec1 None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.domain.id=default, user.id=4a0aec528900484294cd86eb90b642f7) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:41:52.813985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519aecf-e70a-4dc5-81dc-ebd0bd34aec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:52.816054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519aecf-e70a-4dc5-81dc-ebd0bd34aec1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:52.823333 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 28/56] 10.222.0.17 () {56 vars in 1322 bytes} [Mon Jul 25 18:41:52 2022] GET /identity/v3/role_assignments?user.id=4a0aec528900484294cd86eb90b642f7&scope.domain.id=default&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:41:52.902031 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 29/57] 10.222.0.17 () {60 vars in 1140 bytes} [Mon Jul 25 18:41:52 2022] POST /identity/v3/projects => generated 298 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:41:54.938827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba201a-dd06-48df-8b8d-00fa277eafcf None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:54.939372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba201a-dd06-48df-8b8d-00fa277eafcf None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:54.939461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba201a-dd06-48df-8b8d-00fa277eafcf None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:54.940840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 29/58] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:54.946550 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a08cb-744f-4481-a519-94673a18abcf None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:54.946990 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a08cb-744f-4481-a519-94673a18abcf None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:54.947143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a08cb-744f-4481-a519-94673a18abcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:54.965349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c12a08cb-744f-4481-a519-94673a18abcf None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:54.966619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c12a08cb-744f-4481-a519-94673a18abcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:54.980916 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:55.035629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 30/59] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:41:55.060394 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba201a-dd06-48df-8b8d-00fa277eafcf None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:55.164235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba201a-dd06-48df-8b8d-00fa277eafcf None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:55.171347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:55.200913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67547d-5963-44b2-9f0e-f05741f3d4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:55.201760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67547d-5963-44b2-9f0e-f05741f3d4ec None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:55.202047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67547d-5963-44b2-9f0e-f05741f3d4ec None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:55.202342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67547d-5963-44b2-9f0e-f05741f3d4ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:55.203522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67547d-5963-44b2-9f0e-f05741f3d4ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:55.204969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67547d-5963-44b2-9f0e-f05741f3d4ec None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:55.207040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 30/60] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:41:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:41:55.212649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a08cb-744f-4481-a519-94673a18abcf None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:55.320791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a08cb-744f-4481-a519-94673a18abcf None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:55.355748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fd60f-c59e-4a0a-8922-e80f5e721724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:55.356428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fd60f-c59e-4a0a-8922-e80f5e721724 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:55.356717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fd60f-c59e-4a0a-8922-e80f5e721724 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:55.357001 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fd60f-c59e-4a0a-8922-e80f5e721724 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:55.357716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fd60f-c59e-4a0a-8922-e80f5e721724 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:55.359198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fd60f-c59e-4a0a-8922-e80f5e721724 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:55.631610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 31/61] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:41:55 2022] POST /identity/v3/users => generated 275 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:41:59.030838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6e1e7-e9b3-4b59-b9d1-e630f7eecbd9 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.030838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6e1e7-e9b3-4b59-b9d1-e630f7eecbd9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.030838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6e1e7-e9b3-4b59-b9d1-e630f7eecbd9 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.030838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 31/62] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:59.036963 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c6462-3421-4a65-98a4-8af12405ed7c None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.037924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c6462-3421-4a65-98a4-8af12405ed7c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.038458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c6462-3421-4a65-98a4-8af12405ed7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.052249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:59.067637 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3a8c6462-3421-4a65-98a4-8af12405ed7c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:59.071029 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3a8c6462-3421-4a65-98a4-8af12405ed7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:59.170655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 32/63] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:41:59.232176 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6e1e7-e9b3-4b59-b9d1-e630f7eecbd9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:59.284496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d78b4-5cb6-4f88-8f97-b34a9eba9c2b None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.284890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d78b4-5cb6-4f88-8f97-b34a9eba9c2b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.285317 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d78b4-5cb6-4f88-8f97-b34a9eba9c2b None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.287044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 33/64] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:59.292010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a369655-37d2-46db-bad5-fdbc15d26517 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.293324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a369655-37d2-46db-bad5-fdbc15d26517 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.293839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a369655-37d2-46db-bad5-fdbc15d26517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.319734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-0a369655-37d2-46db-bad5-fdbc15d26517 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:59.322114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0a369655-37d2-46db-bad5-fdbc15d26517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:59.366257 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6e1e7-e9b3-4b59-b9d1-e630f7eecbd9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:59.406065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6b535-914a-4cc8-b4e6-f3f9970b86f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:59.407241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6b535-914a-4cc8-b4e6-f3f9970b86f2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.407771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6b535-914a-4cc8-b4e6-f3f9970b86f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.408258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6b535-914a-4cc8-b4e6-f3f9970b86f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.412653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 34/65] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:41:59.417304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ac99e-457f-44fb-b5e0-af87c26c43c2 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.417799 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ac99e-457f-44fb-b5e0-af87c26c43c2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.419862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ac99e-457f-44fb-b5e0-af87c26c43c2 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.422056 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 35/66] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:59.430196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6b535-914a-4cc8-b4e6-f3f9970b86f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:41:59.432056 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294ba5e-1442-46d6-88fe-cec3552db7e1 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.432356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294ba5e-1442-46d6-88fe-cec3552db7e1 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.432644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294ba5e-1442-46d6-88fe-cec3552db7e1 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.434023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 36/67] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:41:59.435183 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d34cb-0f2c-4a31-a93e-dba9c1fb3305 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.435670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d34cb-0f2c-4a31-a93e-dba9c1fb3305 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.436123 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d34cb-0f2c-4a31-a93e-dba9c1fb3305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-97f6b535-914a-4cc8-b4e6-f3f9970b86f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:41:59.445630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:41:59.446593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:41:59.447372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:41:59.447372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:41:59.447372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:41:59.447372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:41:59.447372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:41:59.447372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:41:59.447372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:41:59.447372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:41:59.447372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:41:59.447372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:41:59.447372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:41:59.447372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:41:59.448628 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 32/68] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:41:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:41:59.448832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cdcc7-ea54-4b52-a4b7-92bfb4714de4 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.449129 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cdcc7-ea54-4b52-a4b7-92bfb4714de4 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.449417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cdcc7-ea54-4b52-a4b7-92bfb4714de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.466117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-821d34cb-0f2c-4a31-a93e-dba9c1fb3305 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:59.467559 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-821d34cb-0f2c-4a31-a93e-dba9c1fb3305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:59.472073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d36cdcc7-ea54-4b52-a4b7-92bfb4714de4 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:59.473305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d36cdcc7-ea54-4b52-a4b7-92bfb4714de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:59.508450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:59.535069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:59.544545 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 37/69] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:41:59.545180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d34cb-0f2c-4a31-a93e-dba9c1fb3305 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:59.571544 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 33/70] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:41:59.572641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cdcc7-ea54-4b52-a4b7-92bfb4714de4 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:59.655108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d34cb-0f2c-4a31-a93e-dba9c1fb3305 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:59.677219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cdcc7-ea54-4b52-a4b7-92bfb4714de4 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:59.692543 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557f3b5-b6f2-4484-a7b9-3b06a937790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:59.693457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557f3b5-b6f2-4484-a7b9-3b06a937790e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.693862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557f3b5-b6f2-4484-a7b9-3b06a937790e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.694018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557f3b5-b6f2-4484-a7b9-3b06a937790e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.704941 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e557f3b5-b6f2-4484-a7b9-3b06a937790e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-e557f3b5-b6f2-4484-a7b9-3b06a937790e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:41:59.713668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:41:59.714742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 38/71] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:41:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:41:59.721064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b2898-6bfb-4303-aef9-0de66a932f30 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.721739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b2898-6bfb-4303-aef9-0de66a932f30 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.721739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b2898-6bfb-4303-aef9-0de66a932f30 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.721739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 39/72] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:41:59.721739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b2898-6bfb-4303-aef9-0de66a932f30 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:59.721970 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f71015-dfc8-4e2b-a636-2c2eebcf2213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:59.721970 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f71015-dfc8-4e2b-a636-2c2eebcf2213 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.721970 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f71015-dfc8-4e2b-a636-2c2eebcf2213 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.722954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f71015-dfc8-4e2b-a636-2c2eebcf2213 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.723355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f71015-dfc8-4e2b-a636-2c2eebcf2213 None admin] RBAC: Adding query filter params (name=member) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:41:59.723674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f71015-dfc8-4e2b-a636-2c2eebcf2213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:41:59.725782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f71015-dfc8-4e2b-a636-2c2eebcf2213 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:41:59.736034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 34/73] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:41:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:41:59.736907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f71015-dfc8-4e2b-a636-2c2eebcf2213 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:59.767675 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:59.789560 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:59.837597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b2898-6bfb-4303-aef9-0de66a932f30 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:59.842932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f71015-dfc8-4e2b-a636-2c2eebcf2213 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:41:59.873575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68215e20-412c-42df-bade-32ffe456a634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:59.874428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68215e20-412c-42df-bade-32ffe456a634 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.874722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68215e20-412c-42df-bade-32ffe456a634 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.875003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68215e20-412c-42df-bade-32ffe456a634 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.878085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b459fa-c1dd-4bd9-97bc-9c4d07bdd949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:41:59.879557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b459fa-c1dd-4bd9-97bc-9c4d07bdd949 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.879947 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b459fa-c1dd-4bd9-97bc-9c4d07bdd949 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.880261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b459fa-c1dd-4bd9-97bc-9c4d07bdd949 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.886722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68215e20-412c-42df-bade-32ffe456a634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:41:59.890816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b459fa-c1dd-4bd9-97bc-9c4d07bdd949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-68215e20-412c-42df-bade-32ffe456a634 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:41:59.895232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:41:59.896098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:41:59.896874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:41:59.896874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:41:59.896874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:41:59.896874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:41:59.896874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:41:59.896874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:41:59.896874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:41:59.896874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:41:59.896874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:41:59.896874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:41:59.896874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:41:59.896874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:41:59.896874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 40/74] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:41:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:41:59.898033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b89927-6484-4f60-9c03-b2e09cebd42f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:41:59.898033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b89927-6484-4f60-9c03-b2e09cebd42f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:41:59.898263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b89927-6484-4f60-9c03-b2e09cebd42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-e1b459fa-c1dd-4bd9-97bc-9c4d07bdd949 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:41:59.899308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:41:59.900427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 35/75] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:41:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:41:59.901548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b459fa-c1dd-4bd9-97bc-9c4d07bdd949 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:41:59.919719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c0b89927-6484-4f60-9c03-b2e09cebd42f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:41:59.920960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c0b89927-6484-4f60-9c03-b2e09cebd42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:41:59.991974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:41:59.995926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 41/76] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:41:59.996864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b89927-6484-4f60-9c03-b2e09cebd42f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:00.007770 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b459fa-c1dd-4bd9-97bc-9c4d07bdd949 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:00.043054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef0960-81e6-4ca1-9211-e890b91ca20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:00.043738 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef0960-81e6-4ca1-9211-e890b91ca20d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:00.043944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef0960-81e6-4ca1-9211-e890b91ca20d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:00.044232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef0960-81e6-4ca1-9211-e890b91ca20d None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:00.045126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef0960-81e6-4ca1-9211-e890b91ca20d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:00.045318 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef0960-81e6-4ca1-9211-e890b91ca20d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:00.046752 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef0960-81e6-4ca1-9211-e890b91ca20d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:00.054141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:00.055832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 36/77] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:41:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:00.059830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef0960-81e6-4ca1-9211-e890b91ca20d None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:00.115836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b89927-6484-4f60-9c03-b2e09cebd42f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:00.158382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cde60f-f621-4512-a52e-ecac4c91a2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:00.159201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cde60f-f621-4512-a52e-ecac4c91a2d0 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:00.159675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cde60f-f621-4512-a52e-ecac4c91a2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:00.160001 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cde60f-f621-4512-a52e-ecac4c91a2d0 None admin] PATH_INFO: `/v3/users/04ba33eeeb194cc6b55f828248627dbb` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:00.177332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cde60f-f621-4512-a52e-ecac4c91a2d0 None admin] RBAC: Authorizing `identity:get_user(user_id=04ba33eeeb194cc6b55f828248627dbb)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:00.178919 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cde60f-f621-4512-a52e-ecac4c91a2d0 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:00.180739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 42/78] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:41:59 2022] GET /identity/v3/users/04ba33eeeb194cc6b55f828248627dbb => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:00.181516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cde60f-f621-4512-a52e-ecac4c91a2d0 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:00.182047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef0960-81e6-4ca1-9211-e890b91ca20d None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:00.220801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da844c2b-f2c8-4c04-bf08-07dbc6df6093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:00.221494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da844c2b-f2c8-4c04-bf08-07dbc6df6093 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:00.221786 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da844c2b-f2c8-4c04-bf08-07dbc6df6093 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:00.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da844c2b-f2c8-4c04-bf08-07dbc6df6093 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:00.223458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da844c2b-f2c8-4c04-bf08-07dbc6df6093 None admin] RBAC: Adding query filter params (name=member) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:00.223776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da844c2b-f2c8-4c04-bf08-07dbc6df6093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:00.225180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da844c2b-f2c8-4c04-bf08-07dbc6df6093 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:00.234239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 37/79] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:00.235119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da844c2b-f2c8-4c04-bf08-07dbc6df6093 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:00.250718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:00.292891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cde60f-f621-4512-a52e-ecac4c91a2d0 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:00.294574 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:00.329890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5662f-5c44-4d55-b083-819a51149bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:00.330641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5662f-5c44-4d55-b083-819a51149bdd None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:00.330938 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5662f-5c44-4d55-b083-819a51149bdd None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:00.331224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5662f-5c44-4d55-b083-819a51149bdd None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:00.332283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5662f-5c44-4d55-b083-819a51149bdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:00.332749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5662f-5c44-4d55-b083-819a51149bdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:00.335133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5662f-5c44-4d55-b083-819a51149bdd None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:00.344484 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 43/80] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:00.345657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da844c2b-f2c8-4c04-bf08-07dbc6df6093 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:00.345763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5662f-5c44-4d55-b083-819a51149bdd None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:00.384175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d10af-58a6-4d56-beeb-97576e26ad1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:00.384991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d10af-58a6-4d56-beeb-97576e26ad1a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:00.385274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d10af-58a6-4d56-beeb-97576e26ad1a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:00.385555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d10af-58a6-4d56-beeb-97576e26ad1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:00.386800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d10af-58a6-4d56-beeb-97576e26ad1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:00.388256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d10af-58a6-4d56-beeb-97576e26ad1a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:00.389903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 38/81] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:00.390756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d10af-58a6-4d56-beeb-97576e26ad1a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:00.452834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5662f-5c44-4d55-b083-819a51149bdd None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:00.478849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:00.488790 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e99a4f-c7f6-4e1c-b4a3-ac7d41035cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:00.494610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e99a4f-c7f6-4e1c-b4a3-ac7d41035cee None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:00.494610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e99a4f-c7f6-4e1c-b4a3-ac7d41035cee None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:00.494610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e99a4f-c7f6-4e1c-b4a3-ac7d41035cee None admin] PATH_INFO: `/v3/users/04ba33eeeb194cc6b55f828248627dbb` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:00.494610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e99a4f-c7f6-4e1c-b4a3-ac7d41035cee None admin] RBAC: Authorizing `identity:get_user(user_id=04ba33eeeb194cc6b55f828248627dbb)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:00.494610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e99a4f-c7f6-4e1c-b4a3-ac7d41035cee None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:00.495054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 44/82] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:00 2022] GET /identity/v3/users/04ba33eeeb194cc6b55f828248627dbb => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:00.495547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e99a4f-c7f6-4e1c-b4a3-ac7d41035cee None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:00.503024 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d10af-58a6-4d56-beeb-97576e26ad1a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:00.529399 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:00.538819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3aa02-8ebf-4ef6-9e0e-d28798e8a4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:00.543836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3aa02-8ebf-4ef6-9e0e-d28798e8a4e0 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:00.543836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3aa02-8ebf-4ef6-9e0e-d28798e8a4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:00.543836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3aa02-8ebf-4ef6-9e0e-d28798e8a4e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:00.607447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e99a4f-c7f6-4e1c-b4a3-ac7d41035cee None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:00.615065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3aa02-8ebf-4ef6-9e0e-d28798e8a4e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:00.616571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3aa02-8ebf-4ef6-9e0e-d28798e8a4e0 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:00.643619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98149d57-aa43-4297-b841-5d773d9ef858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:00.644857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98149d57-aa43-4297-b841-5d773d9ef858 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:00.645162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98149d57-aa43-4297-b841-5d773d9ef858 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:00.645453 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98149d57-aa43-4297-b841-5d773d9ef858 None admin] PATH_INFO: `/v3/users/04ba33eeeb194cc6b55f828248627dbb` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:00.646709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98149d57-aa43-4297-b841-5d773d9ef858 None admin] RBAC: Authorizing `identity:get_user(user_id=04ba33eeeb194cc6b55f828248627dbb)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:00.648163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98149d57-aa43-4297-b841-5d773d9ef858 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:00.650002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 45/83] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:00 2022] GET /identity/v3/users/04ba33eeeb194cc6b55f828248627dbb => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:00.651699 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98149d57-aa43-4297-b841-5d773d9ef858 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:00.701835 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:00.726421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 39/84] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:00.731416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3aa02-8ebf-4ef6-9e0e-d28798e8a4e0 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:00.757188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:00.760418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98149d57-aa43-4297-b841-5d773d9ef858 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:00.798083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cc964-69d1-404b-b5b8-3ddc9614a20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:00.799008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cc964-69d1-404b-b5b8-3ddc9614a20f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:00.799296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cc964-69d1-404b-b5b8-3ddc9614a20f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:00.799577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cc964-69d1-404b-b5b8-3ddc9614a20f None admin] PATH_INFO: `/v3/users/4a0aec528900484294cd86eb90b642f7` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:00.801698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cc964-69d1-404b-b5b8-3ddc9614a20f None admin] RBAC: Authorizing `identity:get_user(user_id=4a0aec528900484294cd86eb90b642f7)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:00.803198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cc964-69d1-404b-b5b8-3ddc9614a20f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:00.804885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 46/85] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:00 2022] GET /identity/v3/users/4a0aec528900484294cd86eb90b642f7 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:00.805668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cc964-69d1-404b-b5b8-3ddc9614a20f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:00.844149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3aa02-8ebf-4ef6-9e0e-d28798e8a4e0 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:00.882723 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8d94b-f6b9-4eec-9e78-14b952987efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:00.883322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8d94b-f6b9-4eec-9e78-14b952987efe None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:00.883615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8d94b-f6b9-4eec-9e78-14b952987efe None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:00.883899 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8d94b-f6b9-4eec-9e78-14b952987efe None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:00.884628 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8d94b-f6b9-4eec-9e78-14b952987efe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:00.886103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8d94b-f6b9-4eec-9e78-14b952987efe None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:00.939862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cc964-69d1-404b-b5b8-3ddc9614a20f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:00.974565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efa61d-7db3-477c-ad49-bf3ec866d2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:00.975235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efa61d-7db3-477c-ad49-bf3ec866d2db None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:00.975529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efa61d-7db3-477c-ad49-bf3ec866d2db None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:00.975813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efa61d-7db3-477c-ad49-bf3ec866d2db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:01.012297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:01.015083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 40/86] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:00 2022] POST /identity/v3/users => generated 289 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:01.018937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8d94b-f6b9-4eec-9e78-14b952987efe None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:01.059384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:01.061940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5efa61d-7db3-477c-ad49-bf3ec866d2db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:01.063653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5efa61d-7db3-477c-ad49-bf3ec866d2db None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:01.142904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8d94b-f6b9-4eec-9e78-14b952987efe None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:01.189763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74e497-fe9f-4e8c-a707-6efbc02812d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:01.190717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74e497-fe9f-4e8c-a707-6efbc02812d0 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:01.190872 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74e497-fe9f-4e8c-a707-6efbc02812d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:01.191176 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74e497-fe9f-4e8c-a707-6efbc02812d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:01.194514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 47/87] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:01.195931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efa61d-7db3-477c-ad49-bf3ec866d2db None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:01.279397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:01.302550 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74e497-fe9f-4e8c-a707-6efbc02812d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:01.304093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74e497-fe9f-4e8c-a707-6efbc02812d0 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:01.339510 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efa61d-7db3-477c-ad49-bf3ec866d2db None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:01.356978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:01.446443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f49a0-6c4b-4093-aecb-c792e98ca9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:01.447879 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f49a0-6c4b-4093-aecb-c792e98ca9fb None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:01.448177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f49a0-6c4b-4093-aecb-c792e98ca9fb None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:01.448637 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f49a0-6c4b-4093-aecb-c792e98ca9fb None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:01.450096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f49a0-6c4b-4093-aecb-c792e98ca9fb None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:01.451717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f49a0-6c4b-4093-aecb-c792e98ca9fb None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:01.453841 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 48/88] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:01 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:01.455505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f49a0-6c4b-4093-aecb-c792e98ca9fb None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:01.486497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 41/89] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:01.487503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74e497-fe9f-4e8c-a707-6efbc02812d0 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:01.577808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f49a0-6c4b-4093-aecb-c792e98ca9fb None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:01.595665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74e497-fe9f-4e8c-a707-6efbc02812d0 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:01.618721 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b75f2-ea49-4e8e-ba25-eb1c26b2889b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:01.619872 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b75f2-ea49-4e8e-ba25-eb1c26b2889b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:01.620419 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b75f2-ea49-4e8e-ba25-eb1c26b2889b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:01.620951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b75f2-ea49-4e8e-ba25-eb1c26b2889b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:01.625336 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:01.632832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84868a67-f866-4ef2-b83f-9903736c2a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:01.633722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84868a67-f866-4ef2-b83f-9903736c2a68 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:01.634012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84868a67-f866-4ef2-b83f-9903736c2a68 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:01.634292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84868a67-f866-4ef2-b83f-9903736c2a68 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:01.635777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84868a67-f866-4ef2-b83f-9903736c2a68 None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:01.637726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84868a67-f866-4ef2-b83f-9903736c2a68 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:01.642639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 42/90] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:01 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:01.642639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84868a67-f866-4ef2-b83f-9903736c2a68 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:01.643376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:01.706172 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b75f2-ea49-4e8e-ba25-eb1c26b2889b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:01.708597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b75f2-ea49-4e8e-ba25-eb1c26b2889b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:01.750842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84868a67-f866-4ef2-b83f-9903736c2a68 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:01.797411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc30480-7b75-489c-aca1-03deb6a8ad9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:01.798145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc30480-7b75-489c-aca1-03deb6a8ad9f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:01.798513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc30480-7b75-489c-aca1-03deb6a8ad9f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:01.798847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc30480-7b75-489c-aca1-03deb6a8ad9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:01.799717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc30480-7b75-489c-aca1-03deb6a8ad9f None admin] RBAC: Adding query filter params (role.id=9dc988eb162e47e48df810053eecb165, scope.project.id=f0435acd015c454f926f9318bb5503b2, user.id=04ba33eeeb194cc6b55f828248627dbb) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:01.800078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc30480-7b75-489c-aca1-03deb6a8ad9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:01.802770 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc30480-7b75-489c-aca1-03deb6a8ad9f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:01.816617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 43/91] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:01 2022] GET /identity/v3/role_assignments?user.id=04ba33eeeb194cc6b55f828248627dbb&scope.project.id=f0435acd015c454f926f9318bb5503b2&role.id=9dc988eb162e47e48df810053eecb165 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:01.816617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc30480-7b75-489c-aca1-03deb6a8ad9f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:01.854359 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 49/92] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:01.855862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b75f2-ea49-4e8e-ba25-eb1c26b2889b None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:01.887894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:01.917643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:01.929865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc30480-7b75-489c-aca1-03deb6a8ad9f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:01.978436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085711e8-db28-4746-978d-654d3ac11f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:01.978436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085711e8-db28-4746-978d-654d3ac11f7a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:01.978436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085711e8-db28-4746-978d-654d3ac11f7a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:01.978436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085711e8-db28-4746-978d-654d3ac11f7a None admin] PATH_INFO: `/v3/projects/e691780eee4540e9b1543ad7a438d935` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:01.978823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085711e8-db28-4746-978d-654d3ac11f7a None admin] RBAC: Authorizing `identity:get_project(project_id=e691780eee4540e9b1543ad7a438d935)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:01.978823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085711e8-db28-4746-978d-654d3ac11f7a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:01.978823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 44/93] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:01 2022] GET /identity/v3/projects/e691780eee4540e9b1543ad7a438d935 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:01.978823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085711e8-db28-4746-978d-654d3ac11f7a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:02.102184 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085711e8-db28-4746-978d-654d3ac11f7a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:02.104395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b75f2-ea49-4e8e-ba25-eb1c26b2889b None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:02.142270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e01bd-2079-4e72-a00e-11cb8f5bc030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:02.143565 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e01bd-2079-4e72-a00e-11cb8f5bc030 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:02.144708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e01bd-2079-4e72-a00e-11cb8f5bc030 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:02.145280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e01bd-2079-4e72-a00e-11cb8f5bc030 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:02.147012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e01bd-2079-4e72-a00e-11cb8f5bc030 None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:02.148709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e01bd-2079-4e72-a00e-11cb8f5bc030 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:02.150715 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 45/94] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:01 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:02.151692 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e01bd-2079-4e72-a00e-11cb8f5bc030 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:02.154727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:02.171710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169feacc-18c0-4a31-9d41-541cbe57eba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:02.172412 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169feacc-18c0-4a31-9d41-541cbe57eba0 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:02.172757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169feacc-18c0-4a31-9d41-541cbe57eba0 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:02.173042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169feacc-18c0-4a31-9d41-541cbe57eba0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:02.173886 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169feacc-18c0-4a31-9d41-541cbe57eba0 None admin] RBAC: Adding query filter params (role.id=a638e91b70624f669ffe76dac32323f5, scope.project.id=f0435acd015c454f926f9318bb5503b2, user.id=04ba33eeeb194cc6b55f828248627dbb) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:02.174202 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169feacc-18c0-4a31-9d41-541cbe57eba0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:02.176360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169feacc-18c0-4a31-9d41-541cbe57eba0 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:02.196462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 50/95] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:01 2022] GET /identity/v3/role_assignments?user.id=04ba33eeeb194cc6b55f828248627dbb&scope.project.id=f0435acd015c454f926f9318bb5503b2&role.id=a638e91b70624f669ffe76dac32323f5 => generated 259 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:02.197342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169feacc-18c0-4a31-9d41-541cbe57eba0 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:02.301116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e01bd-2079-4e72-a00e-11cb8f5bc030 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:02.349274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69b894-ee13-4d28-9861-21d88b2a2b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:02.350318 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69b894-ee13-4d28-9861-21d88b2a2b5e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:02.351011 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69b894-ee13-4d28-9861-21d88b2a2b5e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:02.351501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69b894-ee13-4d28-9861-21d88b2a2b5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:02.352842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69b894-ee13-4d28-9861-21d88b2a2b5e None admin] RBAC: Adding query filter params (role.id=9dc988eb162e47e48df810053eecb165, scope.project.id=e691780eee4540e9b1543ad7a438d935, user.id=04ba33eeeb194cc6b55f828248627dbb) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:02.353382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69b894-ee13-4d28-9861-21d88b2a2b5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:02.355186 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69b894-ee13-4d28-9861-21d88b2a2b5e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:02.362303 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 46/96] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:02 2022] GET /identity/v3/role_assignments?user.id=04ba33eeeb194cc6b55f828248627dbb&scope.project.id=e691780eee4540e9b1543ad7a438d935&role.id=9dc988eb162e47e48df810053eecb165 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:02.431062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:02.434966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169feacc-18c0-4a31-9d41-541cbe57eba0 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:02.475395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca0606-9053-4a2e-be05-33224c851efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:02.476539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0606-9053-4a2e-be05-33224c851efb None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:02.477076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0606-9053-4a2e-be05-33224c851efb None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:02.477561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0606-9053-4a2e-be05-33224c851efb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:02.478656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca0606-9053-4a2e-be05-33224c851efb None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=f0435acd015c454f926f9318bb5503b2, user.id=4a0aec528900484294cd86eb90b642f7) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:02.479272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca0606-9053-4a2e-be05-33224c851efb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:02.480949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca0606-9053-4a2e-be05-33224c851efb None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:02.492488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 51/97] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:02 2022] GET /identity/v3/role_assignments?user.id=4a0aec528900484294cd86eb90b642f7&scope.project.id=f0435acd015c454f926f9318bb5503b2&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:03.663175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca5ceb-e639-4135-9dfe-f1307b75532b None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:03.663717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca5ceb-e639-4135-9dfe-f1307b75532b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:03.663794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca5ceb-e639-4135-9dfe-f1307b75532b None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:03.665353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 47/98] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:03.670632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a63769-14fb-451d-891c-f508e92bfa0e None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:03.671047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a63769-14fb-451d-891c-f508e92bfa0e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:03.671221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a63769-14fb-451d-891c-f508e92bfa0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:03.690692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e5a63769-14fb-451d-891c-f508e92bfa0e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:03.692024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e5a63769-14fb-451d-891c-f508e92bfa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:03.787385 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 52/99] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:03.816180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca5ceb-e639-4135-9dfe-f1307b75532b None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:03.891681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:03.930899 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca5ceb-e639-4135-9dfe-f1307b75532b None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:03.967528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e833c-a6ba-4bfe-80ee-47c7ae72f25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:03.968356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e833c-a6ba-4bfe-80ee-47c7ae72f25d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:03.968651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e833c-a6ba-4bfe-80ee-47c7ae72f25d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:03.968940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e833c-a6ba-4bfe-80ee-47c7ae72f25d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:03.983529 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e833c-a6ba-4bfe-80ee-47c7ae72f25d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-810e833c-a6ba-4bfe-80ee-47c7ae72f25d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:03.994785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:03.995672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:03.997701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:03.997701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:03.997701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:03.997701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:03.997701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:03.997701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:03.997701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:03.997701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:03.997701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:03.997701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:03.997701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:03.997701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:03.997701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 48/100] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:04.002126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a63769-14fb-451d-891c-f508e92bfa0e None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:04.121861 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:04.129333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a63769-14fb-451d-891c-f508e92bfa0e None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:04.141638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c350ce5-462a-4ed9-a6ac-e20c7bf537f6 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:04.141941 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c350ce5-462a-4ed9-a6ac-e20c7bf537f6 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:04.142229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c350ce5-462a-4ed9-a6ac-e20c7bf537f6 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:04.143789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 49/101] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:04.149966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff801420-dca4-4bf7-96aa-7beaf45ad07c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:04.150357 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff801420-dca4-4bf7-96aa-7beaf45ad07c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:04.150649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff801420-dca4-4bf7-96aa-7beaf45ad07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:04.194296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ff801420-dca4-4bf7-96aa-7beaf45ad07c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:04.195484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ff801420-dca4-4bf7-96aa-7beaf45ad07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:04.225644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07beb59-f77f-49dc-a4ef-c9f487720a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:04.226438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07beb59-f77f-49dc-a4ef-c9f487720a98 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:04.226737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07beb59-f77f-49dc-a4ef-c9f487720a98 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:04.227025 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07beb59-f77f-49dc-a4ef-c9f487720a98 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:04.227800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07beb59-f77f-49dc-a4ef-c9f487720a98 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:04.228119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07beb59-f77f-49dc-a4ef-c9f487720a98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:04.231459 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07beb59-f77f-49dc-a4ef-c9f487720a98 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:04.251484 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 53/102] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:04.258107 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07beb59-f77f-49dc-a4ef-c9f487720a98 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:04.376102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 50/103] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 25 18:42:04.416912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff801420-dca4-4bf7-96aa-7beaf45ad07c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:04.470227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07beb59-f77f-49dc-a4ef-c9f487720a98 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:04.513820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347fdcb-0515-461d-b6fc-5a281b6e115f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:04.514749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347fdcb-0515-461d-b6fc-5a281b6e115f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:04.515042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347fdcb-0515-461d-b6fc-5a281b6e115f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:04.515326 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347fdcb-0515-461d-b6fc-5a281b6e115f None admin] PATH_INFO: `/v3/users/1dc98c5904dd482eb49bd3a04afe6d28` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:04.547038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:04.555976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347fdcb-0515-461d-b6fc-5a281b6e115f None admin] RBAC: Authorizing `identity:get_user(user_id=1dc98c5904dd482eb49bd3a04afe6d28)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:04.557619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347fdcb-0515-461d-b6fc-5a281b6e115f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:04.559855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 54/104] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:04 2022] GET /identity/v3/users/1dc98c5904dd482eb49bd3a04afe6d28 => generated 289 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:04.565164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347fdcb-0515-461d-b6fc-5a281b6e115f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:04.598978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff801420-dca4-4bf7-96aa-7beaf45ad07c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:04.602355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:04.639012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e142a2-8f69-48cc-8fda-fb0b04b6d073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:04.640156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e142a2-8f69-48cc-8fda-fb0b04b6d073 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:04.640630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e142a2-8f69-48cc-8fda-fb0b04b6d073 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:04.641061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e142a2-8f69-48cc-8fda-fb0b04b6d073 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:04.642759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e142a2-8f69-48cc-8fda-fb0b04b6d073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:04.645384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e142a2-8f69-48cc-8fda-fb0b04b6d073 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:04.647665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 51/105] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:04.657679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e142a2-8f69-48cc-8fda-fb0b04b6d073 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:04.776615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e142a2-8f69-48cc-8fda-fb0b04b6d073 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:04.812004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:04.818085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27204e8e-a1c3-482b-809d-61704e21c054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:04.819113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27204e8e-a1c3-482b-809d-61704e21c054 None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:04.819581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27204e8e-a1c3-482b-809d-61704e21c054 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:04.820073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27204e8e-a1c3-482b-809d-61704e21c054 None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:04.821065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27204e8e-a1c3-482b-809d-61704e21c054 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:04.822850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27204e8e-a1c3-482b-809d-61704e21c054 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:04.844532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347fdcb-0515-461d-b6fc-5a281b6e115f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:04.886203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a407aa0-d87e-4263-a2c3-03e1a0a23c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:04.887073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a407aa0-d87e-4263-a2c3-03e1a0a23c21 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:04.887373 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a407aa0-d87e-4263-a2c3-03e1a0a23c21 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:04.887662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a407aa0-d87e-4263-a2c3-03e1a0a23c21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:04.924003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 52/106] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:42:04 2022] POST /identity/v3/projects => generated 302 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:42:04.925707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff4f3b-b292-4636-a823-09fe66139f68 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:04.926155 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff4f3b-b292-4636-a823-09fe66139f68 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:04.926609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff4f3b-b292-4636-a823-09fe66139f68 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:04.936015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 53/107] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:04 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:04.956003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff6e9f-6b54-4fa0-94c9-c781e6c0a073 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:04.956534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff6e9f-6b54-4fa0-94c9-c781e6c0a073 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:04.957739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff6e9f-6b54-4fa0-94c9-c781e6c0a073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:04.974268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a407aa0-d87e-4263-a2c3-03e1a0a23c21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:04.975852 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a407aa0-d87e-4263-a2c3-03e1a0a23c21 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:04.993205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:04.998507 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-5aff6e9f-6b54-4fa0-94c9-c781e6c0a073 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:05.000027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5aff6e9f-6b54-4fa0-94c9-c781e6c0a073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:05.096319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 55/108] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 18:42:05.098138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a407aa0-d87e-4263-a2c3-03e1a0a23c21 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:05.154851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 54/109] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:05.234386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a407aa0-d87e-4263-a2c3-03e1a0a23c21 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:05.240822 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff6e9f-6b54-4fa0-94c9-c781e6c0a073 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:05.250785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:05.273462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae34d7-6af1-423c-9150-4aba80aaf9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:05.274395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae34d7-6af1-423c-9150-4aba80aaf9d3 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:05.274691 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae34d7-6af1-423c-9150-4aba80aaf9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:05.274983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae34d7-6af1-423c-9150-4aba80aaf9d3 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:05.276397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae34d7-6af1-423c-9150-4aba80aaf9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:05.277889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae34d7-6af1-423c-9150-4aba80aaf9d3 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:05.279704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 56/110] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:05 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:05.286401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae34d7-6af1-423c-9150-4aba80aaf9d3 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:05.402607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae34d7-6af1-423c-9150-4aba80aaf9d3 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:05.407106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff6e9f-6b54-4fa0-94c9-c781e6c0a073 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:05.415859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:05.450099 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72334d-4fbd-4e7c-8c84-dcae1adb3137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:05.451034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72334d-4fbd-4e7c-8c84-dcae1adb3137 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:05.451327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72334d-4fbd-4e7c-8c84-dcae1adb3137 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:05.451611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72334d-4fbd-4e7c-8c84-dcae1adb3137 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:05.455310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4de02a-ce74-475d-846f-740ae6f4fe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:05.455992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de02a-ce74-475d-846f-740ae6f4fe1d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:05.456286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de02a-ce74-475d-846f-740ae6f4fe1d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:05.456575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de02a-ce74-475d-846f-740ae6f4fe1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:05.458038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4de02a-ce74-475d-846f-740ae6f4fe1d None admin] RBAC: Adding query filter params (role.id=b2f299f9533944ca821a102e13bd4c69, scope.project.id=f0435acd015c454f926f9318bb5503b2, user.id=1dc98c5904dd482eb49bd3a04afe6d28) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:05.459214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4de02a-ce74-475d-846f-740ae6f4fe1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:05.460742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4de02a-ce74-475d-846f-740ae6f4fe1d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:05.464743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d72334d-4fbd-4e7c-8c84-dcae1adb3137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-3d72334d-4fbd-4e7c-8c84-dcae1adb3137 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:05.473181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:05.475435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:05.477458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:05.477458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:05.477458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:05.477458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:05.477458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:05.477458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:05.477458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:05.477458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:05.477458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:05.477458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:05.477458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:05.477458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:05.486673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 57/111] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:05 2022] GET /identity/v3/role_assignments?user.id=1dc98c5904dd482eb49bd3a04afe6d28&scope.project.id=f0435acd015c454f926f9318bb5503b2&role.id=b2f299f9533944ca821a102e13bd4c69 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:05.486983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 55/112] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:05.487676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4de02a-ce74-475d-846f-740ae6f4fe1d None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:05.546620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:05.625635 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4de02a-ce74-475d-846f-740ae6f4fe1d None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:05.675121 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c2eaf-066a-410e-bc24-f3313dbab5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:05.676525 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c2eaf-066a-410e-bc24-f3313dbab5de None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:05.677045 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c2eaf-066a-410e-bc24-f3313dbab5de None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:05.677465 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c2eaf-066a-410e-bc24-f3313dbab5de None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:05.678243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c2eaf-066a-410e-bc24-f3313dbab5de None admin] RBAC: Adding query filter params (name=member) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:05.678590 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c2eaf-066a-410e-bc24-f3313dbab5de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:05.680041 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c2eaf-066a-410e-bc24-f3313dbab5de None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:05.694268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 58/113] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:05.700100 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c2eaf-066a-410e-bc24-f3313dbab5de None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:05.700238 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e0981-7a56-4c9d-8650-5d17992f8b2d None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:05.700317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e0981-7a56-4c9d-8650-5d17992f8b2d None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:05.700946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e0981-7a56-4c9d-8650-5d17992f8b2d None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:05.703373 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 56/114] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:05.711745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5cad2-025c-4550-a7f0-cb84a082efb7 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:05.712098 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5cad2-025c-4550-a7f0-cb84a082efb7 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:05.712319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5cad2-025c-4550-a7f0-cb84a082efb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:05.761849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-caf5cad2-025c-4550-a7f0-cb84a082efb7 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:05.763492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-caf5cad2-025c-4550-a7f0-cb84a082efb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:05.826596 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c2eaf-066a-410e-bc24-f3313dbab5de None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:05.837192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:05.867296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0271cf-49e2-4e5a-8b77-0a78d212cd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:05.868969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0271cf-49e2-4e5a-8b77-0a78d212cd12 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:05.869555 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0271cf-49e2-4e5a-8b77-0a78d212cd12 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:05.870038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0271cf-49e2-4e5a-8b77-0a78d212cd12 None admin] PATH_INFO: `/v3/users/04ba33eeeb194cc6b55f828248627dbb` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:05.871615 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0271cf-49e2-4e5a-8b77-0a78d212cd12 None admin] RBAC: Authorizing `identity:get_user(user_id=04ba33eeeb194cc6b55f828248627dbb)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:05.873343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0271cf-49e2-4e5a-8b77-0a78d212cd12 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:05.875478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 59/115] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:05 2022] GET /identity/v3/users/04ba33eeeb194cc6b55f828248627dbb => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:05.877232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 57/116] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:05.882740 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0271cf-49e2-4e5a-8b77-0a78d212cd12 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:05.936409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182e7d6-1fbf-43c7-bd3f-57d9db00abbc None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:05.936724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182e7d6-1fbf-43c7-bd3f-57d9db00abbc None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:05.936985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182e7d6-1fbf-43c7-bd3f-57d9db00abbc None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:05.938612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 58/117] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:05.941748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182e7d6-1fbf-43c7-bd3f-57d9db00abbc None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:06.022796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0271cf-49e2-4e5a-8b77-0a78d212cd12 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:06.064030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1db3f-c645-42d7-8aff-8261b3dd32c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:06.065426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1db3f-c645-42d7-8aff-8261b3dd32c9 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:06.067091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1db3f-c645-42d7-8aff-8261b3dd32c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:06.067435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1db3f-c645-42d7-8aff-8261b3dd32c9 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:06.069212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1db3f-c645-42d7-8aff-8261b3dd32c9 None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:06.070825 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1db3f-c645-42d7-8aff-8261b3dd32c9 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:06.071557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:06.073887 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 60/118] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:05 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:06.075425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8078018-58a1-4f3c-870f-680a596e1bec None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:06.075876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8078018-58a1-4f3c-870f-680a596e1bec None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:06.076370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8078018-58a1-4f3c-870f-680a596e1bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:06.079098 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182e7d6-1fbf-43c7-bd3f-57d9db00abbc None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:06.096161 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-f8078018-58a1-4f3c-870f-680a596e1bec None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:06.097651 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f8078018-58a1-4f3c-870f-680a596e1bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:06.120538 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef525a8d-75d9-4c0e-aa98-b5e46c4dc910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:06.121371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef525a8d-75d9-4c0e-aa98-b5e46c4dc910 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:06.121662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef525a8d-75d9-4c0e-aa98-b5e46c4dc910 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:06.121950 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef525a8d-75d9-4c0e-aa98-b5e46c4dc910 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:06.137900 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef525a8d-75d9-4c0e-aa98-b5e46c4dc910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:06.146101 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-ef525a8d-75d9-4c0e-aa98-b5e46c4dc910 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:06.148463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:06.149486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:06.150482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:06.150482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:06.150482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:06.150482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:06.150482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:06.150482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:06.150482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:06.150482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:06.150482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:06.150482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:06.150482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:06.150482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:06.152494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 59/119] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:06.153944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b799f-ac53-44e9-8b9c-0e69a0886d67 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:06.154553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b799f-ac53-44e9-8b9c-0e69a0886d67 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:06.160514 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b799f-ac53-44e9-8b9c-0e69a0886d67 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:06.160804 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 60/120] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:06.162161 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b799f-ac53-44e9-8b9c-0e69a0886d67 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:06.178820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 61/121] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:06.178820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8078018-58a1-4f3c-870f-680a596e1bec None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:06.304399 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8078018-58a1-4f3c-870f-680a596e1bec None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:06.336596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b799f-ac53-44e9-8b9c-0e69a0886d67 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:06.345930 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cd4ce-3784-48f3-a86f-aa80e354a06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:06.347016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd4ce-3784-48f3-a86f-aa80e354a06d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:06.347393 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd4ce-3784-48f3-a86f-aa80e354a06d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:06.348071 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd4ce-3784-48f3-a86f-aa80e354a06d None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:06.349253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cd4ce-3784-48f3-a86f-aa80e354a06d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:06.349682 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cd4ce-3784-48f3-a86f-aa80e354a06d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:06.351529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cd4ce-3784-48f3-a86f-aa80e354a06d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:06.361961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 62/122] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:06.364119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7422570-07df-4aad-87b9-850f3f13b0ff None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:06.364507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7422570-07df-4aad-87b9-850f3f13b0ff None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:06.364895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7422570-07df-4aad-87b9-850f3f13b0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:06.385281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0a0ba-7fc7-4eee-832c-bef0e83b733a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:06.386620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0a0ba-7fc7-4eee-832c-bef0e83b733a None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:06.387052 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0a0ba-7fc7-4eee-832c-bef0e83b733a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:06.387315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b7422570-07df-4aad-87b9-850f3f13b0ff None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:06.387739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0a0ba-7fc7-4eee-832c-bef0e83b733a None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2/users/04ba33eeeb194cc6b55f828248627dbb/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:06.388746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b7422570-07df-4aad-87b9-850f3f13b0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:06.390204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0a0ba-7fc7-4eee-832c-bef0e83b733a None admin] RBAC: Authorizing `identity:create_grant(project_id=f0435acd015c454f926f9318bb5503b2, user_id=04ba33eeeb194cc6b55f828248627dbb, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:06.393164 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0a0ba-7fc7-4eee-832c-bef0e83b733a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:06.430695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:06.450072 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 61/123] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:06 2022] PUT /identity/v3/projects/f0435acd015c454f926f9318bb5503b2/users/04ba33eeeb194cc6b55f828248627dbb/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:06.451268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0a0ba-7fc7-4eee-832c-bef0e83b733a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:06.501567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:06.509720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 63/124] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:06.516569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7422570-07df-4aad-87b9-850f3f13b0ff None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:06.694264 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0a0ba-7fc7-4eee-832c-bef0e83b733a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:06.720176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7422570-07df-4aad-87b9-850f3f13b0ff None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:06.741758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15160f9c-697d-42d2-8a5b-dd29807ef8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:06.743222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15160f9c-697d-42d2-8a5b-dd29807ef8f3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:06.743719 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15160f9c-697d-42d2-8a5b-dd29807ef8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:06.744263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15160f9c-697d-42d2-8a5b-dd29807ef8f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:06.757872 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15160f9c-697d-42d2-8a5b-dd29807ef8f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:06.759456 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa46cab-9f90-4884-80ab-835faef0f22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:06.760370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa46cab-9f90-4884-80ab-835faef0f22a None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:06.760661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa46cab-9f90-4884-80ab-835faef0f22a None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:06.760944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa46cab-9f90-4884-80ab-835faef0f22a None admin] PATH_INFO: `/v3/users/04ba33eeeb194cc6b55f828248627dbb` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:06.762979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa46cab-9f90-4884-80ab-835faef0f22a None admin] RBAC: Authorizing `identity:get_user(user_id=04ba33eeeb194cc6b55f828248627dbb)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:06.764460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa46cab-9f90-4884-80ab-835faef0f22a None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:06.766163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 64/125] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:06 2022] GET /identity/v3/users/04ba33eeeb194cc6b55f828248627dbb => generated 275 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:06.766981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa46cab-9f90-4884-80ab-835faef0f22a None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-15160f9c-697d-42d2-8a5b-dd29807ef8f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:06.768313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:06.769170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:06.770889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:06.770889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:06.770889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:06.770889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:06.770889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:06.770889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:06.770889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:06.770889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:06.770889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:06.770889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:06.770889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:06.770889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:06.773422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 62/126] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:06.774653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15160f9c-697d-42d2-8a5b-dd29807ef8f3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:06.807106 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:06.877517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:06.897108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa46cab-9f90-4884-80ab-835faef0f22a None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:06.905509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15160f9c-697d-42d2-8a5b-dd29807ef8f3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:06.943148 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99596f3-1799-4eb3-a059-536a593768cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:06.944447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99596f3-1799-4eb3-a059-536a593768cf None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:06.944943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99596f3-1799-4eb3-a059-536a593768cf None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:06.945424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99596f3-1799-4eb3-a059-536a593768cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:06.954355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5e508-622b-4928-b128-57ac855bf523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:06.954355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5e508-622b-4928-b128-57ac855bf523 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:06.954355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5e508-622b-4928-b128-57ac855bf523 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:06.954355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5e508-622b-4928-b128-57ac855bf523 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:06.954730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5e508-622b-4928-b128-57ac855bf523 None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:06.954730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5e508-622b-4928-b128-57ac855bf523 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:06.954730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 63/127] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:06 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:06.954730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5e508-622b-4928-b128-57ac855bf523 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:06.970436 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99596f3-1799-4eb3-a059-536a593768cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-b99596f3-1799-4eb3-a059-536a593768cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:06.986302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:06.987248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:06.988114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:06.988114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:06.988114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:06.988114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:06.988114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:06.988114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:06.988114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:06.988114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:06.988114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:06.988114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:06.988114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:06.988114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:07.003248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 65/128] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:07.004784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99596f3-1799-4eb3-a059-536a593768cf None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:07.135548 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:07.143115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99596f3-1799-4eb3-a059-536a593768cf None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:07.153419 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5e508-622b-4928-b128-57ac855bf523 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:07.198514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0facc8b-40ca-4a5e-b75a-3e033424b8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:07.199363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0facc8b-40ca-4a5e-b75a-3e033424b8a2 None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:07.199664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0facc8b-40ca-4a5e-b75a-3e033424b8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:07.199954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0facc8b-40ca-4a5e-b75a-3e033424b8a2 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2/users/04ba33eeeb194cc6b55f828248627dbb/roles/a638e91b70624f669ffe76dac32323f5` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:07.205029 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0facc8b-40ca-4a5e-b75a-3e033424b8a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0435acd015c454f926f9318bb5503b2, user_id=04ba33eeeb194cc6b55f828248627dbb, role_id=a638e91b70624f669ffe76dac32323f5)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:07.207451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0facc8b-40ca-4a5e-b75a-3e033424b8a2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:07.211879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75216a92-ac51-407e-9a6a-fca901693151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:07.212591 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75216a92-ac51-407e-9a6a-fca901693151 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:07.212885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75216a92-ac51-407e-9a6a-fca901693151 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:07.213199 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75216a92-ac51-407e-9a6a-fca901693151 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:07.213949 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75216a92-ac51-407e-9a6a-fca901693151 None admin] RBAC: Adding query filter params (name=member) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:07.214273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75216a92-ac51-407e-9a6a-fca901693151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:07.218415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75216a92-ac51-407e-9a6a-fca901693151 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:07.227771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 64/129] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:07.228418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75216a92-ac51-407e-9a6a-fca901693151 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:07.238309 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:07.295971 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 66/130] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:07 2022] PUT /identity/v3/projects/f0435acd015c454f926f9318bb5503b2/users/04ba33eeeb194cc6b55f828248627dbb/roles/a638e91b70624f669ffe76dac32323f5 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:07.297132 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0facc8b-40ca-4a5e-b75a-3e033424b8a2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:07.341863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75216a92-ac51-407e-9a6a-fca901693151 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:07.386415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791e10c-76dd-4c5f-af89-dd7af20216a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:07.386415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791e10c-76dd-4c5f-af89-dd7af20216a7 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:07.386415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791e10c-76dd-4c5f-af89-dd7af20216a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:07.386415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791e10c-76dd-4c5f-af89-dd7af20216a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:07.386840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791e10c-76dd-4c5f-af89-dd7af20216a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:07.386840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791e10c-76dd-4c5f-af89-dd7af20216a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:07.387148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791e10c-76dd-4c5f-af89-dd7af20216a7 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:07.398210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 65/131] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:07.404165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791e10c-76dd-4c5f-af89-dd7af20216a7 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:07.500877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:07.512347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0facc8b-40ca-4a5e-b75a-3e033424b8a2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:07.544617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791e10c-76dd-4c5f-af89-dd7af20216a7 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:07.586466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45260d-573e-44f4-806d-60a1377667ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:07.587492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45260d-573e-44f4-806d-60a1377667ea None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:07.587799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45260d-573e-44f4-806d-60a1377667ea None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:07.588093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45260d-573e-44f4-806d-60a1377667ea None admin] PATH_INFO: `/v3/users/4a0aec528900484294cd86eb90b642f7` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:07.589391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45260d-573e-44f4-806d-60a1377667ea None admin] RBAC: Authorizing `identity:get_user(user_id=4a0aec528900484294cd86eb90b642f7)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:07.591005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45260d-573e-44f4-806d-60a1377667ea None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:07.593850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 66/132] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:07 2022] GET /identity/v3/users/4a0aec528900484294cd86eb90b642f7 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:07.599984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45260d-573e-44f4-806d-60a1377667ea None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:07.615112 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96790986-bc61-4877-9500-14682b3e5a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:07.616062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96790986-bc61-4877-9500-14682b3e5a12 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:07.616418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96790986-bc61-4877-9500-14682b3e5a12 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:07.616701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96790986-bc61-4877-9500-14682b3e5a12 None admin] PATH_INFO: `/v3/users/04ba33eeeb194cc6b55f828248627dbb` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:07.618008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96790986-bc61-4877-9500-14682b3e5a12 None admin] RBAC: Authorizing `identity:get_user(user_id=04ba33eeeb194cc6b55f828248627dbb)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:07.619506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96790986-bc61-4877-9500-14682b3e5a12 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:07.621393 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 67/133] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:07 2022] GET /identity/v3/users/04ba33eeeb194cc6b55f828248627dbb => generated 275 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:07.627508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96790986-bc61-4877-9500-14682b3e5a12 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:07.676988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:07.717463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45260d-573e-44f4-806d-60a1377667ea None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:07.776266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec6dae-7925-4cf9-8c58-791f2c3a2f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:07.777081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec6dae-7925-4cf9-8c58-791f2c3a2f46 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:07.777372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec6dae-7925-4cf9-8c58-791f2c3a2f46 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:07.777656 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec6dae-7925-4cf9-8c58-791f2c3a2f46 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:07.779071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec6dae-7925-4cf9-8c58-791f2c3a2f46 None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:07.783020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec6dae-7925-4cf9-8c58-791f2c3a2f46 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:07.788675 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 67/134] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:07 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:07.794866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec6dae-7925-4cf9-8c58-791f2c3a2f46 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:07.801579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:07.820753 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96790986-bc61-4877-9500-14682b3e5a12 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:07.931783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec6dae-7925-4cf9-8c58-791f2c3a2f46 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:07.947820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ec524-1541-4caf-83d1-9a5676c529a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:07.948867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ec524-1541-4caf-83d1-9a5676c529a7 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:07.949160 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ec524-1541-4caf-83d1-9a5676c529a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:07.949445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ec524-1541-4caf-83d1-9a5676c529a7 None admin] PATH_INFO: `/v3/projects/e691780eee4540e9b1543ad7a438d935` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:07.950741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ec524-1541-4caf-83d1-9a5676c529a7 None admin] RBAC: Authorizing `identity:get_project(project_id=e691780eee4540e9b1543ad7a438d935)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:07.952208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ec524-1541-4caf-83d1-9a5676c529a7 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:07.953949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 68/135] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:07 2022] GET /identity/v3/projects/e691780eee4540e9b1543ad7a438d935 => generated 312 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:07.963322 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ec524-1541-4caf-83d1-9a5676c529a7 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:07.968893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426e816-6e8d-4627-a15b-decd08484020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:07.969814 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426e816-6e8d-4627-a15b-decd08484020 None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:07.970110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426e816-6e8d-4627-a15b-decd08484020 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:07.970432 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426e816-6e8d-4627-a15b-decd08484020 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2/users/4a0aec528900484294cd86eb90b642f7/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:07.975248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426e816-6e8d-4627-a15b-decd08484020 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0435acd015c454f926f9318bb5503b2, user_id=4a0aec528900484294cd86eb90b642f7, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:07.976193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426e816-6e8d-4627-a15b-decd08484020 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:08.049681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 68/136] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:07 2022] PUT /identity/v3/projects/f0435acd015c454f926f9318bb5503b2/users/4a0aec528900484294cd86eb90b642f7/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:08.129957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:08.141091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ec524-1541-4caf-83d1-9a5676c529a7 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:08.173180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fd89f-1220-48c4-aa12-e0977de7d39d None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:08.173715 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fd89f-1220-48c4-aa12-e0977de7d39d None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:08.174147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fd89f-1220-48c4-aa12-e0977de7d39d None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:08.176342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 69/137] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:08.183559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31f1e5-cd0b-4cef-a215-2f9d59d5e534 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:08.184089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31f1e5-cd0b-4cef-a215-2f9d59d5e534 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:08.184671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31f1e5-cd0b-4cef-a215-2f9d59d5e534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:08.202911 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897f4ef-9520-4e5b-8d49-0f4693806972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:08.204175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897f4ef-9520-4e5b-8d49-0f4693806972 None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:08.204638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897f4ef-9520-4e5b-8d49-0f4693806972 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:08.205101 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897f4ef-9520-4e5b-8d49-0f4693806972 None admin] PATH_INFO: `/v3/projects/e691780eee4540e9b1543ad7a438d935/users/04ba33eeeb194cc6b55f828248627dbb/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:08.209296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897f4ef-9520-4e5b-8d49-0f4693806972 None admin] RBAC: Authorizing `identity:create_grant(project_id=e691780eee4540e9b1543ad7a438d935, user_id=04ba33eeeb194cc6b55f828248627dbb, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:08.211248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897f4ef-9520-4e5b-8d49-0f4693806972 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:08.217540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ce31f1e5-cd0b-4cef-a215-2f9d59d5e534 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:08.219203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ce31f1e5-cd0b-4cef-a215-2f9d59d5e534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:08.289338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 69/138] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:07 2022] PUT /identity/v3/projects/e691780eee4540e9b1543ad7a438d935/users/04ba33eeeb194cc6b55f828248627dbb/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:08.297637 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568256f-e77c-4ad0-8e5a-6ef9ecab7d7a None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:08.297966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568256f-e77c-4ad0-8e5a-6ef9ecab7d7a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:08.298179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568256f-e77c-4ad0-8e5a-6ef9ecab7d7a None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:08.300020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 70/139] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:08.304089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:08.312165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29e46e-dea1-4e35-a4e5-da93aced9d8d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:08.312481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29e46e-dea1-4e35-a4e5-da93aced9d8d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:08.312761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29e46e-dea1-4e35-a4e5-da93aced9d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:08.334919 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 70/140] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:08.342220 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ae29e46e-dea1-4e35-a4e5-da93aced9d8d None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:08.343552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ae29e46e-dea1-4e35-a4e5-da93aced9d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:08.388841 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31f1e5-cd0b-4cef-a215-2f9d59d5e534 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:08.473561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 71/141] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:08.515046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29e46e-dea1-4e35-a4e5-da93aced9d8d None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:08.533152 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31f1e5-cd0b-4cef-a215-2f9d59d5e534 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:08.571799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737d8b2-125b-4af8-807d-43bd36a7329f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:08.572553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737d8b2-125b-4af8-807d-43bd36a7329f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:08.572851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737d8b2-125b-4af8-807d-43bd36a7329f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:08.573139 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737d8b2-125b-4af8-807d-43bd36a7329f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:08.585438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737d8b2-125b-4af8-807d-43bd36a7329f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-1737d8b2-125b-4af8-807d-43bd36a7329f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:08.593661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:08.594704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:08.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:08.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:08.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:08.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:08.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:08.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:08.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:08.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:08.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:08.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:08.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:08.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:08.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 71/142] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:08.604832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737d8b2-125b-4af8-807d-43bd36a7329f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:08.674727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:08.693252 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:08.733506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29e46e-dea1-4e35-a4e5-da93aced9d8d None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:08.748497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737d8b2-125b-4af8-807d-43bd36a7329f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:08.811511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8b960-e972-40f0-9943-be8adce42c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:08.812642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8b960-e972-40f0-9943-be8adce42c69 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:08.813867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8b960-e972-40f0-9943-be8adce42c69 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:08.814411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8b960-e972-40f0-9943-be8adce42c69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:08.816619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8b960-e972-40f0-9943-be8adce42c69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:08.818828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8b960-e972-40f0-9943-be8adce42c69 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:08.828843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d86ae0-2b86-4082-af75-af9bce21da57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:08.829823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d86ae0-2b86-4082-af75-af9bce21da57 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:08.830015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 72/143] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:08.830498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d86ae0-2b86-4082-af75-af9bce21da57 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:08.830968 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d86ae0-2b86-4082-af75-af9bce21da57 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:08.832607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8b960-e972-40f0-9943-be8adce42c69 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:08.833585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d86ae0-2b86-4082-af75-af9bce21da57 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:08.834121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d86ae0-2b86-4082-af75-af9bce21da57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:08.838799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d86ae0-2b86-4082-af75-af9bce21da57 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:08.858990 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 72/144] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:08.867406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d86ae0-2b86-4082-af75-af9bce21da57 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:09.038478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8b960-e972-40f0-9943-be8adce42c69 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:09.043316 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d86ae0-2b86-4082-af75-af9bce21da57 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:09.074706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:09.080889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9429c-c5da-4f1c-b6ab-cdb63690c9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:09.081648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9429c-c5da-4f1c-b6ab-cdb63690c9a8 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:09.082675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9429c-c5da-4f1c-b6ab-cdb63690c9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:09.083063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9429c-c5da-4f1c-b6ab-cdb63690c9a8 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:09.083881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9429c-c5da-4f1c-b6ab-cdb63690c9a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:09.085417 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9429c-c5da-4f1c-b6ab-cdb63690c9a8 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:09.108355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63910f-77e6-49f3-bac6-221848cfcd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:09.109701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63910f-77e6-49f3-bac6-221848cfcd5f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:09.110056 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63910f-77e6-49f3-bac6-221848cfcd5f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:09.110465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63910f-77e6-49f3-bac6-221848cfcd5f None admin] PATH_INFO: `/v3/users/1dc98c5904dd482eb49bd3a04afe6d28` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:09.111875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63910f-77e6-49f3-bac6-221848cfcd5f None admin] RBAC: Authorizing `identity:get_user(user_id=1dc98c5904dd482eb49bd3a04afe6d28)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:09.113611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63910f-77e6-49f3-bac6-221848cfcd5f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:09.115541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 73/145] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:08 2022] GET /identity/v3/users/1dc98c5904dd482eb49bd3a04afe6d28 => generated 289 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:09.122107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63910f-77e6-49f3-bac6-221848cfcd5f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:09.271864 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63910f-77e6-49f3-bac6-221848cfcd5f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:09.302176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 73/146] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:08 2022] POST /identity/v3/users => generated 283 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:09.330051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368feaa-998f-479b-91aa-b7c4eedb344d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:09.331173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368feaa-998f-479b-91aa-b7c4eedb344d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:09.331464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368feaa-998f-479b-91aa-b7c4eedb344d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:09.331816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368feaa-998f-479b-91aa-b7c4eedb344d None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:09.333426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368feaa-998f-479b-91aa-b7c4eedb344d None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:09.335095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368feaa-998f-479b-91aa-b7c4eedb344d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:09.337487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 74/147] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:09 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:09.347647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9429c-c5da-4f1c-b6ab-cdb63690c9a8 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:09.357919 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:09.546096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9429c-c5da-4f1c-b6ab-cdb63690c9a8 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:09.597274 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1e635-c58c-48f0-bef5-45f34de33cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:09.598881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1e635-c58c-48f0-bef5-45f34de33cc4 None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:09.599202 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1e635-c58c-48f0-bef5-45f34de33cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:09.599493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1e635-c58c-48f0-bef5-45f34de33cc4 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2/users/1dc98c5904dd482eb49bd3a04afe6d28/roles/b2f299f9533944ca821a102e13bd4c69` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:09.608085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1e635-c58c-48f0-bef5-45f34de33cc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0435acd015c454f926f9318bb5503b2, user_id=1dc98c5904dd482eb49bd3a04afe6d28, role_id=b2f299f9533944ca821a102e13bd4c69)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:09.609655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1e635-c58c-48f0-bef5-45f34de33cc4 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:09.635027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c97b8-8d2c-432b-a0cb-4996f171c220 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:09.635540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c97b8-8d2c-432b-a0cb-4996f171c220 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:09.636061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c97b8-8d2c-432b-a0cb-4996f171c220 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:09.640477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 75/148] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:09.661242 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1ccf5-29af-42aa-b145-bec26db3c6f9 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:09.661770 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1ccf5-29af-42aa-b145-bec26db3c6f9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:09.662132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1ccf5-29af-42aa-b145-bec26db3c6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:09.685799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-4bb1ccf5-29af-42aa-b145-bec26db3c6f9 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:09.687289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-4bb1ccf5-29af-42aa-b145-bec26db3c6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:09.708962 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 74/149] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:09 2022] PUT /identity/v3/projects/f0435acd015c454f926f9318bb5503b2/users/1dc98c5904dd482eb49bd3a04afe6d28/roles/b2f299f9533944ca821a102e13bd4c69 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:09.774698 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:09.800256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 76/150] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:09.846495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1e635-c58c-48f0-bef5-45f34de33cc4 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:09.951396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:10.021200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1e635-c58c-48f0-bef5-45f34de33cc4 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:10.098712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec5d2d-ec7c-45de-a63f-79a2aeddda19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:10.099697 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec5d2d-ec7c-45de-a63f-79a2aeddda19 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:10.099863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec5d2d-ec7c-45de-a63f-79a2aeddda19 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:10.100187 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec5d2d-ec7c-45de-a63f-79a2aeddda19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:10.120898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec5d2d-ec7c-45de-a63f-79a2aeddda19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-64ec5d2d-ec7c-45de-a63f-79a2aeddda19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:10.135266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:10.136786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:10.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:10.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:10.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:10.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:10.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:10.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:10.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:10.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:10.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:10.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:10.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:10.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:10.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 75/151] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:10.147574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1ccf5-29af-42aa-b145-bec26db3c6f9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:10.327974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1ccf5-29af-42aa-b145-bec26db3c6f9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:10.395399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907d5d4-6295-4a29-8c8c-e2b8d903f997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:10.396121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907d5d4-6295-4a29-8c8c-e2b8d903f997 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:10.396437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907d5d4-6295-4a29-8c8c-e2b8d903f997 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:10.396723 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907d5d4-6295-4a29-8c8c-e2b8d903f997 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:10.397615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907d5d4-6295-4a29-8c8c-e2b8d903f997 None admin] RBAC: Adding query filter params (name=member) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:10.398020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907d5d4-6295-4a29-8c8c-e2b8d903f997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:10.399511 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907d5d4-6295-4a29-8c8c-e2b8d903f997 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:10.416636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 77/152] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:10.423111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec5d2d-ec7c-45de-a63f-79a2aeddda19 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:10.607285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec5d2d-ec7c-45de-a63f-79a2aeddda19 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:10.649700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:10.666021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8d630-a039-40d1-9040-b6cf1f5ad96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:10.667605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8d630-a039-40d1-9040-b6cf1f5ad96c None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:10.667904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8d630-a039-40d1-9040-b6cf1f5ad96c None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:10.668192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8d630-a039-40d1-9040-b6cf1f5ad96c None admin] PATH_INFO: `/v3/users/04ba33eeeb194cc6b55f828248627dbb` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:10.671659 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8d630-a039-40d1-9040-b6cf1f5ad96c None admin] RBAC: Authorizing `identity:get_user(user_id=04ba33eeeb194cc6b55f828248627dbb)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:10.673251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8d630-a039-40d1-9040-b6cf1f5ad96c None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:10.675083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 76/153] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:10 2022] GET /identity/v3/users/04ba33eeeb194cc6b55f828248627dbb => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:10.682596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907d5d4-6295-4a29-8c8c-e2b8d903f997 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:10.770980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:10.836191 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03033793-cd49-4209-b052-1fa5863ad601 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:10.836827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03033793-cd49-4209-b052-1fa5863ad601 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:10.837382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03033793-cd49-4209-b052-1fa5863ad601 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:10.843836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 77/154] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:10.855611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907d5d4-6295-4a29-8c8c-e2b8d903f997 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:10.855819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08af56-62af-4bfd-b99b-2199fef1d90f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:10.856433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08af56-62af-4bfd-b99b-2199fef1d90f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:10.856929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08af56-62af-4bfd-b99b-2199fef1d90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:10.896777 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-7b08af56-62af-4bfd-b99b-2199fef1d90f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:10.899422 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-7b08af56-62af-4bfd-b99b-2199fef1d90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:10.903098 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3d921-3144-4a6f-a5f2-281edda807bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:10.903869 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3d921-3144-4a6f-a5f2-281edda807bb None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:10.904157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3d921-3144-4a6f-a5f2-281edda807bb None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:10.904443 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3d921-3144-4a6f-a5f2-281edda807bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:11.024913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3d921-3144-4a6f-a5f2-281edda807bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:11.026475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3d921-3144-4a6f-a5f2-281edda807bb None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:11.067310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 78/155] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:11.124532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:11.171744 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08af56-62af-4bfd-b99b-2199fef1d90f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:11.241708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 78/156] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:11.247715 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3d921-3144-4a6f-a5f2-281edda807bb None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:11.340857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08af56-62af-4bfd-b99b-2199fef1d90f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:11.377003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:11.403222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a00b6e-f4a0-468a-9b14-b42e9d80e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:11.404079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a00b6e-f4a0-468a-9b14-b42e9d80e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:11.404383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a00b6e-f4a0-468a-9b14-b42e9d80e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:11.404742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a00b6e-f4a0-468a-9b14-b42e9d80e9e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:11.427707 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a00b6e-f4a0-468a-9b14-b42e9d80e9e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-52a00b6e-f4a0-468a-9b14-b42e9d80e9e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:11.436808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:11.437762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:11.438553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:11.438553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:11.438553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:11.438553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:11.438553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:11.438553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:11.438553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:11.438553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:11.438553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:11.438553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:11.438553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:11.438553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:11.438553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 79/157] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:11.444952 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a00b6e-f4a0-468a-9b14-b42e9d80e9e4 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:11.523845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3d921-3144-4a6f-a5f2-281edda807bb None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:11.587852 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77975e11-c641-48b6-bacc-7bc2e2e6ffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:11.588861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77975e11-c641-48b6-bacc-7bc2e2e6ffb1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:11.589155 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77975e11-c641-48b6-bacc-7bc2e2e6ffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:11.589942 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77975e11-c641-48b6-bacc-7bc2e2e6ffb1 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:11.591331 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77975e11-c641-48b6-bacc-7bc2e2e6ffb1 None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:11.592851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77975e11-c641-48b6-bacc-7bc2e2e6ffb1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:11.598637 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 79/158] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:11 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:11.601588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77975e11-c641-48b6-bacc-7bc2e2e6ffb1 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:11.627803 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:11.715190 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a00b6e-f4a0-468a-9b14-b42e9d80e9e4 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:11.739771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77975e11-c641-48b6-bacc-7bc2e2e6ffb1 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:11.768260 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a34c8-3109-4617-bf3a-9d226256cafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:11.768966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a34c8-3109-4617-bf3a-9d226256cafb None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:11.769260 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a34c8-3109-4617-bf3a-9d226256cafb None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:11.770103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a34c8-3109-4617-bf3a-9d226256cafb None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:11.770901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a34c8-3109-4617-bf3a-9d226256cafb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:11.771222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a34c8-3109-4617-bf3a-9d226256cafb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:11.772645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a34c8-3109-4617-bf3a-9d226256cafb None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:11.782163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 80/159] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:11.794221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a34c8-3109-4617-bf3a-9d226256cafb None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:11.803103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f8b28-6421-4b7b-87d6-d7617cf4be24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:11.803801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f8b28-6421-4b7b-87d6-d7617cf4be24 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:11.804093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f8b28-6421-4b7b-87d6-d7617cf4be24 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:11.804374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f8b28-6421-4b7b-87d6-d7617cf4be24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:11.805239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f8b28-6421-4b7b-87d6-d7617cf4be24 None admin] RBAC: Adding query filter params (role.id=9dc988eb162e47e48df810053eecb165, scope.project.id=f0435acd015c454f926f9318bb5503b2, user.id=04ba33eeeb194cc6b55f828248627dbb) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:11.805549 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f8b28-6421-4b7b-87d6-d7617cf4be24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:11.807067 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f8b28-6421-4b7b-87d6-d7617cf4be24 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:11.818489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 80/160] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:11 2022] GET /identity/v3/role_assignments?user.id=04ba33eeeb194cc6b55f828248627dbb&scope.project.id=f0435acd015c454f926f9318bb5503b2&role.id=9dc988eb162e47e48df810053eecb165 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:11.867195 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:11.945025 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a34c8-3109-4617-bf3a-9d226256cafb None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:11.992521 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eeb027-5837-4f80-bc83-72db6a78fba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:11.993348 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eeb027-5837-4f80-bc83-72db6a78fba0 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:11.993640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eeb027-5837-4f80-bc83-72db6a78fba0 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:11.993926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eeb027-5837-4f80-bc83-72db6a78fba0 None admin] PATH_INFO: `/v3/users/04ba33eeeb194cc6b55f828248627dbb` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:11.995227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eeb027-5837-4f80-bc83-72db6a78fba0 None admin] RBAC: Authorizing `identity:get_user(user_id=04ba33eeeb194cc6b55f828248627dbb)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:11.996693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eeb027-5837-4f80-bc83-72db6a78fba0 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:11.998566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 81/161] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:11 2022] GET /identity/v3/users/04ba33eeeb194cc6b55f828248627dbb => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:12.004706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f8b28-6421-4b7b-87d6-d7617cf4be24 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:12.105176 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:12.135532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f8b28-6421-4b7b-87d6-d7617cf4be24 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:12.176409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d45f0-87dc-4b1f-8632-f2bbe704e0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:12.177097 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d45f0-87dc-4b1f-8632-f2bbe704e0dc None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:12.177390 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d45f0-87dc-4b1f-8632-f2bbe704e0dc None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:12.177677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d45f0-87dc-4b1f-8632-f2bbe704e0dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:12.281266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0f7eb-bc00-45fd-90be-93f1b0a112a9 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:12.281266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0f7eb-bc00-45fd-90be-93f1b0a112a9 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:12.281266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0f7eb-bc00-45fd-90be-93f1b0a112a9 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:12.281266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 82/162] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:12.288252 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd8116-09ec-49d5-a77b-8c0c614ee42a None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:12.288543 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd8116-09ec-49d5-a77b-8c0c614ee42a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:12.288939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd8116-09ec-49d5-a77b-8c0c614ee42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:12.311534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d45f0-87dc-4b1f-8632-f2bbe704e0dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:12.313190 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d45f0-87dc-4b1f-8632-f2bbe704e0dc None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:12.327611 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-adbd8116-09ec-49d5-a77b-8c0c614ee42a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:12.328952 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-adbd8116-09ec-49d5-a77b-8c0c614ee42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:12.446838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 83/163] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:12.469532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b249c1-9415-4588-9c36-589a792259f0 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:12.469851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b249c1-9415-4588-9c36-589a792259f0 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:12.470183 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b249c1-9415-4588-9c36-589a792259f0 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:12.472013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 84/164] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:12.477631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b249c1-9415-4588-9c36-589a792259f0 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:12.512645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:12.541399 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:12.594724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 81/165] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 18:42:12.596597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ae43a-6dff-4383-9bbb-8456f49905fd None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:12.596909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ae43a-6dff-4383-9bbb-8456f49905fd None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:12.597176 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ae43a-6dff-4383-9bbb-8456f49905fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:12.635486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d19ae43a-6dff-4383-9bbb-8456f49905fd None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:12.636781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d19ae43a-6dff-4383-9bbb-8456f49905fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:12.651544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b249c1-9415-4588-9c36-589a792259f0 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:12.711219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3725daa-fa2b-42d1-9da0-84b4bc155aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:12.712162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3725daa-fa2b-42d1-9da0-84b4bc155aef None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:12.712450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3725daa-fa2b-42d1-9da0-84b4bc155aef None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:12.712733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3725daa-fa2b-42d1-9da0-84b4bc155aef None admin] PATH_INFO: `/v3/roles/member` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:12.729182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3725daa-fa2b-42d1-9da0-84b4bc155aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-b3725daa-fa2b-42d1-9da0-84b4bc155aef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:12.737019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:12.737895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:12.739606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:12.739606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:12.739606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:12.739606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:12.739606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:12.739606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:12.739606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:12.739606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:12.739606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:12.739606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:12.739606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:12.739606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:12.739606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 85/166] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:12.747155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3725daa-fa2b-42d1-9da0-84b4bc155aef None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:12.758103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 82/167] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:12.759090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ae43a-6dff-4383-9bbb-8456f49905fd None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:12.911163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ae43a-6dff-4383-9bbb-8456f49905fd None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:12.920090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:12.955844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4982dfd-e64c-4c44-b12d-c6512a33e036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:12.956803 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4982dfd-e64c-4c44-b12d-c6512a33e036 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:12.958556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4982dfd-e64c-4c44-b12d-c6512a33e036 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:12.958994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4982dfd-e64c-4c44-b12d-c6512a33e036 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:12.959922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4982dfd-e64c-4c44-b12d-c6512a33e036 None admin] RBAC: Adding query filter params (name=member) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:12.961694 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4982dfd-e64c-4c44-b12d-c6512a33e036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:12.964182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4982dfd-e64c-4c44-b12d-c6512a33e036 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:12.973234 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3725daa-fa2b-42d1-9da0-84b4bc155aef None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:12.978419 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:12.980436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 83/168] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:12.982442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4982dfd-e64c-4c44-b12d-c6512a33e036 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:13.063053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe92ef-2e18-4e14-ac0f-57a14f1a0cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:13.063952 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe92ef-2e18-4e14-ac0f-57a14f1a0cc9 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:13.064239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe92ef-2e18-4e14-ac0f-57a14f1a0cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:13.064519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe92ef-2e18-4e14-ac0f-57a14f1a0cc9 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:13.075408 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe92ef-2e18-4e14-ac0f-57a14f1a0cc9 None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:13.076918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe92ef-2e18-4e14-ac0f-57a14f1a0cc9 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:13.086558 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 86/169] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:12 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:13.087490 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe92ef-2e18-4e14-ac0f-57a14f1a0cc9 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:13.297116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4982dfd-e64c-4c44-b12d-c6512a33e036 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:13.379698 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd6565-6160-4548-9869-587c5e3e33c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:13.381307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd6565-6160-4548-9869-587c5e3e33c6 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:13.381745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd6565-6160-4548-9869-587c5e3e33c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:13.382168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd6565-6160-4548-9869-587c5e3e33c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:13.408955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd6565-6160-4548-9869-587c5e3e33c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-39fd6565-6160-4548-9869-587c5e3e33c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:13.422875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:13.424073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:13.424984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:13.424984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:13.424984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:13.424984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:13.424984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:13.424984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:13.424984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:13.424984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:13.424984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:13.424984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:13.424984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:13.424984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:13.427714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 84/170] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 446 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:13.428727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd6565-6160-4548-9869-587c5e3e33c6 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:13.524712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe92ef-2e18-4e14-ac0f-57a14f1a0cc9 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:13.531892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:13.657042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba57517-356c-4ac6-8664-670cc79689e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:13.657868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba57517-356c-4ac6-8664-670cc79689e3 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:13.658157 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba57517-356c-4ac6-8664-670cc79689e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:13.659059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba57517-356c-4ac6-8664-670cc79689e3 None admin] PATH_INFO: `/v3/users/04ba33eeeb194cc6b55f828248627dbb` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:13.661014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba57517-356c-4ac6-8664-670cc79689e3 None admin] RBAC: Authorizing `identity:get_user(user_id=04ba33eeeb194cc6b55f828248627dbb)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:13.667100 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba57517-356c-4ac6-8664-670cc79689e3 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:13.669159 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 87/171] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:13 2022] GET /identity/v3/users/04ba33eeeb194cc6b55f828248627dbb => generated 275 bytes in 582 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:13.670341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba57517-356c-4ac6-8664-670cc79689e3 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:13.744142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd6565-6160-4548-9869-587c5e3e33c6 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:13.801390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:13.816330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f3a3f-6d18-491f-990e-0e14628a71c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:13.819622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f3a3f-6d18-491f-990e-0e14628a71c1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:13.822282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f3a3f-6d18-491f-990e-0e14628a71c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:13.824207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f3a3f-6d18-491f-990e-0e14628a71c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:13.825350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f3a3f-6d18-491f-990e-0e14628a71c1 None admin] RBAC: Adding query filter params (role.id=a638e91b70624f669ffe76dac32323f5, scope.project.id=f0435acd015c454f926f9318bb5503b2, user.id=04ba33eeeb194cc6b55f828248627dbb) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:13.827631 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f3a3f-6d18-491f-990e-0e14628a71c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:13.829364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f3a3f-6d18-491f-990e-0e14628a71c1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:13.859848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 85/172] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:13 2022] GET /identity/v3/role_assignments?user.id=04ba33eeeb194cc6b55f828248627dbb&scope.project.id=f0435acd015c454f926f9318bb5503b2&role.id=a638e91b70624f669ffe76dac32323f5 => generated 608 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:13.862953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f3a3f-6d18-491f-990e-0e14628a71c1 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:14.056144 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba57517-356c-4ac6-8664-670cc79689e3 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:14.128586 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948469e0-1a98-438e-aea0-6ad01f8065a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:14.130779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948469e0-1a98-438e-aea0-6ad01f8065a6 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:14.131306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948469e0-1a98-438e-aea0-6ad01f8065a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:14.131786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948469e0-1a98-438e-aea0-6ad01f8065a6 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:14.132759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948469e0-1a98-438e-aea0-6ad01f8065a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:14.133308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948469e0-1a98-438e-aea0-6ad01f8065a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:14.135525 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948469e0-1a98-438e-aea0-6ad01f8065a6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:14.148522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 88/173] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 479 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:14.150986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b436a-70fd-40a6-a6a0-d0de741f5bdd None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:14.151531 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b436a-70fd-40a6-a6a0-d0de741f5bdd None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:14.152013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b436a-70fd-40a6-a6a0-d0de741f5bdd None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:14.154234 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:14.156353 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 89/174] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:14.158609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ae410-89ad-4836-ac41-3cac06839faf None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:14.159454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ae410-89ad-4836-ac41-3cac06839faf None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:14.160006 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ae410-89ad-4836-ac41-3cac06839faf None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:14.163645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 90/175] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:14.164731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ae410-89ad-4836-ac41-3cac06839faf None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:14.166221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f3a3f-6d18-491f-990e-0e14628a71c1 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:14.218964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec879df-8312-46e4-a8b4-1be7bf5dd9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:14.219740 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec879df-8312-46e4-a8b4-1be7bf5dd9db None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:14.220105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec879df-8312-46e4-a8b4-1be7bf5dd9db None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:14.220604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec879df-8312-46e4-a8b4-1be7bf5dd9db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:14.317591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:14.322079 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec879df-8312-46e4-a8b4-1be7bf5dd9db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:14.322508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ae410-89ad-4836-ac41-3cac06839faf None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:14.324069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec879df-8312-46e4-a8b4-1be7bf5dd9db None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:14.378909 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cf8d4-7605-4f2e-a3aa-1adbfa5f6509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:14.380935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cf8d4-7605-4f2e-a3aa-1adbfa5f6509 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:14.381426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cf8d4-7605-4f2e-a3aa-1adbfa5f6509 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:14.381867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cf8d4-7605-4f2e-a3aa-1adbfa5f6509 None admin] PATH_INFO: `/v3/users/4a0aec528900484294cd86eb90b642f7` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:14.384044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cf8d4-7605-4f2e-a3aa-1adbfa5f6509 None admin] RBAC: Authorizing `identity:get_user(user_id=4a0aec528900484294cd86eb90b642f7)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:14.385736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cf8d4-7605-4f2e-a3aa-1adbfa5f6509 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:14.388818 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 91/176] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:14 2022] GET /identity/v3/users/4a0aec528900484294cd86eb90b642f7 => generated 247 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:14.390981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f5390-2ee1-44a6-9148-2015ed3c397e None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:14.391472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f5390-2ee1-44a6-9148-2015ed3c397e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:14.392390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f5390-2ee1-44a6-9148-2015ed3c397e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:14.418841 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-006f5390-2ee1-44a6-9148-2015ed3c397e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:14.421315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-006f5390-2ee1-44a6-9148-2015ed3c397e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:14.448097 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:14.465993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 86/177] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:14.467800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b58f99-6e27-44e1-aff7-dccb64bee62c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:14.468697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b58f99-6e27-44e1-aff7-dccb64bee62c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:14.468985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b58f99-6e27-44e1-aff7-dccb64bee62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:14.490122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-15b58f99-6e27-44e1-aff7-dccb64bee62c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:14.492114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-15b58f99-6e27-44e1-aff7-dccb64bee62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:14.504425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 92/178] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:14.505235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f5390-2ee1-44a6-9148-2015ed3c397e None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:14.583560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 87/179] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:14.584837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d696f-1536-4a13-baf1-0a12a15153fe None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:14.585130 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d696f-1536-4a13-baf1-0a12a15153fe None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:14.585414 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d696f-1536-4a13-baf1-0a12a15153fe None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:14.586790 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 88/180] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:14.587534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d696f-1536-4a13-baf1-0a12a15153fe None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:14.633464 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:14.641613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f5390-2ee1-44a6-9148-2015ed3c397e None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:14.680798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2df69-a0d0-439f-a014-5364f1f300ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:14.681464 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2df69-a0d0-439f-a014-5364f1f300ef None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:14.681752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2df69-a0d0-439f-a014-5364f1f300ef None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:14.682038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2df69-a0d0-439f-a014-5364f1f300ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:14.703750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d696f-1536-4a13-baf1-0a12a15153fe None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:14.727757 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:14.741022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443afff-cd86-4e6a-a53a-6ca7964ab79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:14.743416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443afff-cd86-4e6a-a53a-6ca7964ab79d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:14.743416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443afff-cd86-4e6a-a53a-6ca7964ab79d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:14.743416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443afff-cd86-4e6a-a53a-6ca7964ab79d None admin] PATH_INFO: `/v3/projects/e691780eee4540e9b1543ad7a438d935` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:14.744489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443afff-cd86-4e6a-a53a-6ca7964ab79d None admin] RBAC: Authorizing `identity:get_project(project_id=e691780eee4540e9b1543ad7a438d935)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:14.746250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443afff-cd86-4e6a-a53a-6ca7964ab79d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:14.753085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 89/181] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:14 2022] GET /identity/v3/projects/e691780eee4540e9b1543ad7a438d935 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:14.753085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e875f-7d38-43c2-9eb8-70e89957dc4c None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:14.753085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e875f-7d38-43c2-9eb8-70e89957dc4c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:14.753085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e875f-7d38-43c2-9eb8-70e89957dc4c None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:14.753085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 90/182] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:14.753085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e875f-7d38-43c2-9eb8-70e89957dc4c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:14.761955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2df69-a0d0-439f-a014-5364f1f300ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:14.763485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2df69-a0d0-439f-a014-5364f1f300ef None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:14.866960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:14.872159 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 93/183] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:14.873334 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e875f-7d38-43c2-9eb8-70e89957dc4c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:14.873611 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8cdb7-a861-4c13-83c2-df758ca8aee9 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:14.873898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8cdb7-a861-4c13-83c2-df758ca8aee9 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:14.874214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8cdb7-a861-4c13-83c2-df758ca8aee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:14.893567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-4ff8cdb7-a861-4c13-83c2-df758ca8aee9 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:14.901241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4ff8cdb7-a861-4c13-83c2-df758ca8aee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:14.914145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224602e-6b58-4f94-9c25-1a90a719a173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:14.915058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224602e-6b58-4f94-9c25-1a90a719a173 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:14.915439 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224602e-6b58-4f94-9c25-1a90a719a173 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:14.915749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224602e-6b58-4f94-9c25-1a90a719a173 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:14.927040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224602e-6b58-4f94-9c25-1a90a719a173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-e224602e-6b58-4f94-9c25-1a90a719a173 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:14.934849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:14.935812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:14.936613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:14.936613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:14.936613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:14.936613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:14.936613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:14.936613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:14.936613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:14.936613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:14.936613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:14.936613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:14.936613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:14.936613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:14.936613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 91/184] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:14.937837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224602e-6b58-4f94-9c25-1a90a719a173 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:14.971566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 94/185] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:14.972496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8cdb7-a861-4c13-83c2-df758ca8aee9 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:15.004644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:15.063927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224602e-6b58-4f94-9c25-1a90a719a173 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:15.078583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8cdb7-a861-4c13-83c2-df758ca8aee9 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:15.100841 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810c17b-80c0-448b-8c8f-cb231afae048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:15.101647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810c17b-80c0-448b-8c8f-cb231afae048 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:15.101937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810c17b-80c0-448b-8c8f-cb231afae048 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:15.102220 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810c17b-80c0-448b-8c8f-cb231afae048 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:15.107605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:15.113292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810c17b-80c0-448b-8c8f-cb231afae048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:15.120983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67916f0-fa33-43b2-ba90-05c1370e061a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:15.120983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67916f0-fa33-43b2-ba90-05c1370e061a None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:15.120983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67916f0-fa33-43b2-ba90-05c1370e061a None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:15.120983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67916f0-fa33-43b2-ba90-05c1370e061a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:15.121362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67916f0-fa33-43b2-ba90-05c1370e061a None admin] RBAC: Adding query filter params (role.id=9dc988eb162e47e48df810053eecb165, scope.project.id=e691780eee4540e9b1543ad7a438d935, user.id=04ba33eeeb194cc6b55f828248627dbb) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:15.121362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67916f0-fa33-43b2-ba90-05c1370e061a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:15.121869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67916f0-fa33-43b2-ba90-05c1370e061a None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-d810c17b-80c0-448b-8c8f-cb231afae048 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:15.122278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:15.123125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:15.124231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:15.124231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:15.124231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:15.124231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:15.124231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:15.124231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:15.124231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:15.124231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:15.124231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:15.124231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:15.124231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:15.124231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:15.124231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 92/186] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:15.125980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b938629-ebbb-4bd2-b5a7-048d8250acda None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:15.126410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b938629-ebbb-4bd2-b5a7-048d8250acda None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:15.126802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b938629-ebbb-4bd2-b5a7-048d8250acda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:15.129583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 95/187] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:14 2022] GET /identity/v3/role_assignments?user.id=04ba33eeeb194cc6b55f828248627dbb&scope.project.id=e691780eee4540e9b1543ad7a438d935&role.id=9dc988eb162e47e48df810053eecb165 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:15.131101 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67916f0-fa33-43b2-ba90-05c1370e061a None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:15.147898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-7b938629-ebbb-4bd2-b5a7-048d8250acda None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:15.149158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7b938629-ebbb-4bd2-b5a7-048d8250acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:15.241463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 93/188] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:15.242399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b938629-ebbb-4bd2-b5a7-048d8250acda None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:15.284837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67916f0-fa33-43b2-ba90-05c1370e061a None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:15.306217 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:15.329830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8d6f1-e77b-4f2f-ad5a-074f52aed003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:15.330754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8d6f1-e77b-4f2f-ad5a-074f52aed003 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:15.331046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8d6f1-e77b-4f2f-ad5a-074f52aed003 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:15.331328 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8d6f1-e77b-4f2f-ad5a-074f52aed003 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:15.333512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8d6f1-e77b-4f2f-ad5a-074f52aed003 None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:15.334989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8d6f1-e77b-4f2f-ad5a-074f52aed003 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:15.339048 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 96/189] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:15 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:15.339048 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8d6f1-e77b-4f2f-ad5a-074f52aed003 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:15.384102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b938629-ebbb-4bd2-b5a7-048d8250acda None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:15.408459 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:15.419621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e762e76-76ef-4b53-8690-dd59b8b4e072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:15.420311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e762e76-76ef-4b53-8690-dd59b8b4e072 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:15.420599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e762e76-76ef-4b53-8690-dd59b8b4e072 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:15.420885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e762e76-76ef-4b53-8690-dd59b8b4e072 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:15.421641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e762e76-76ef-4b53-8690-dd59b8b4e072 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:15.421953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e762e76-76ef-4b53-8690-dd59b8b4e072 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:15.423382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e762e76-76ef-4b53-8690-dd59b8b4e072 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:15.432814 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 94/190] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:15.433649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e762e76-76ef-4b53-8690-dd59b8b4e072 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:15.456943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8d6f1-e77b-4f2f-ad5a-074f52aed003 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:15.495386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c99f6-97c3-44a1-bdad-c26b703178c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:15.496280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c99f6-97c3-44a1-bdad-c26b703178c1 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:15.496563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c99f6-97c3-44a1-bdad-c26b703178c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:15.496841 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c99f6-97c3-44a1-bdad-c26b703178c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:15.508717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c99f6-97c3-44a1-bdad-c26b703178c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-f03c99f6-97c3-44a1-bdad-c26b703178c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:15.516994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:15.517946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:15.518785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:15.518785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:15.518785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:15.518785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:15.518785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:15.518785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:15.518785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:15.518785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:15.518785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:15.518785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:15.518785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:15.518785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:15.518785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 97/191] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:15.519439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c99f6-97c3-44a1-bdad-c26b703178c1 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:15.543344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e762e76-76ef-4b53-8690-dd59b8b4e072 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:15.544998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:15.593388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747fbcf-6812-47b0-94b4-691299d8a780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:15.594083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747fbcf-6812-47b0-94b4-691299d8a780 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:15.594401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747fbcf-6812-47b0-94b4-691299d8a780 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:15.594690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747fbcf-6812-47b0-94b4-691299d8a780 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:15.595456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747fbcf-6812-47b0-94b4-691299d8a780 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:15.595765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747fbcf-6812-47b0-94b4-691299d8a780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:15.597428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747fbcf-6812-47b0-94b4-691299d8a780 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:15.607223 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 95/192] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:15.608078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747fbcf-6812-47b0-94b4-691299d8a780 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:15.651985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c99f6-97c3-44a1-bdad-c26b703178c1 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:15.672962 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:15.689607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d038d-b769-4fe9-a424-d1fa1185b7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:15.690832 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d038d-b769-4fe9-a424-d1fa1185b7ed None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:15.691267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d038d-b769-4fe9-a424-d1fa1185b7ed None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:15.692280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d038d-b769-4fe9-a424-d1fa1185b7ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:15.694478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d038d-b769-4fe9-a424-d1fa1185b7ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:15.696203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d038d-b769-4fe9-a424-d1fa1185b7ed None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:15.698046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 98/193] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:15.698877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d038d-b769-4fe9-a424-d1fa1185b7ed None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:15.728831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747fbcf-6812-47b0-94b4-691299d8a780 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:15.762836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63729d67-2c99-4b47-8d5c-98922c8abc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:15.763503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63729d67-2c99-4b47-8d5c-98922c8abc71 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:15.763795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63729d67-2c99-4b47-8d5c-98922c8abc71 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:15.764077 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63729d67-2c99-4b47-8d5c-98922c8abc71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:15.764912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63729d67-2c99-4b47-8d5c-98922c8abc71 None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=f0435acd015c454f926f9318bb5503b2, user.id=4a0aec528900484294cd86eb90b642f7) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:15.765224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63729d67-2c99-4b47-8d5c-98922c8abc71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:15.766702 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63729d67-2c99-4b47-8d5c-98922c8abc71 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:15.773213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 96/194] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:15 2022] GET /identity/v3/role_assignments?user.id=4a0aec528900484294cd86eb90b642f7&scope.project.id=f0435acd015c454f926f9318bb5503b2&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:15.774031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63729d67-2c99-4b47-8d5c-98922c8abc71 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:15.814797 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d038d-b769-4fe9-a424-d1fa1185b7ed None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:15.818471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:15.856726 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177387a2-7290-4600-823f-8a8985028df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:15.857637 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177387a2-7290-4600-823f-8a8985028df5 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:15.857934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177387a2-7290-4600-823f-8a8985028df5 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:15.858222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177387a2-7290-4600-823f-8a8985028df5 None admin] PATH_INFO: `/v3/users/1dc98c5904dd482eb49bd3a04afe6d28` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:15.859585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177387a2-7290-4600-823f-8a8985028df5 None admin] RBAC: Authorizing `identity:get_user(user_id=1dc98c5904dd482eb49bd3a04afe6d28)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:15.861119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177387a2-7290-4600-823f-8a8985028df5 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:15.862961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 99/195] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:15 2022] GET /identity/v3/users/1dc98c5904dd482eb49bd3a04afe6d28 => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:15.864055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177387a2-7290-4600-823f-8a8985028df5 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:15.921268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63729d67-2c99-4b47-8d5c-98922c8abc71 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:15.933652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:15.977097 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e751d-15d2-45e3-bbfd-d0d4d2fc7d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:15.977793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e751d-15d2-45e3-bbfd-d0d4d2fc7d2a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:15.978880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e751d-15d2-45e3-bbfd-d0d4d2fc7d2a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:15.978880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e751d-15d2-45e3-bbfd-d0d4d2fc7d2a None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:15.979423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e751d-15d2-45e3-bbfd-d0d4d2fc7d2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:15.979423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e751d-15d2-45e3-bbfd-d0d4d2fc7d2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:15.980850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e751d-15d2-45e3-bbfd-d0d4d2fc7d2a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:15.990858 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 97/196] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:15.992051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e751d-15d2-45e3-bbfd-d0d4d2fc7d2a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:16.006398 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177387a2-7290-4600-823f-8a8985028df5 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:16.052829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56a288-4e47-44c3-a19d-d4c943d25862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:16.053717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56a288-4e47-44c3-a19d-d4c943d25862 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.054009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56a288-4e47-44c3-a19d-d4c943d25862 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.054293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56a288-4e47-44c3-a19d-d4c943d25862 None admin] PATH_INFO: `/v3/users/f8c83e570c724b1f97ec66b5d5233ecb` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.068849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56a288-4e47-44c3-a19d-d4c943d25862 None admin] RBAC: Authorizing `identity:get_user(user_id=f8c83e570c724b1f97ec66b5d5233ecb)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:16.070360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56a288-4e47-44c3-a19d-d4c943d25862 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:16.072085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 100/197] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:15 2022] GET /identity/v3/users/f8c83e570c724b1f97ec66b5d5233ecb => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:16.072861 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56a288-4e47-44c3-a19d-d4c943d25862 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:16.109208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:16.119503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e751d-15d2-45e3-bbfd-d0d4d2fc7d2a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:16.156903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350caaa1-dc96-43fd-8729-7a6dff694a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:16.157594 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350caaa1-dc96-43fd-8729-7a6dff694a1f None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.157883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350caaa1-dc96-43fd-8729-7a6dff694a1f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.158165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350caaa1-dc96-43fd-8729-7a6dff694a1f None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.158921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350caaa1-dc96-43fd-8729-7a6dff694a1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:16.160494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350caaa1-dc96-43fd-8729-7a6dff694a1f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:16.187872 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56a288-4e47-44c3-a19d-d4c943d25862 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:16.204496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:16.233295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f076eb3-f214-448e-8647-04f8e42f52e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:16.233973 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f076eb3-f214-448e-8647-04f8e42f52e0 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.234629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f076eb3-f214-448e-8647-04f8e42f52e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.234629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f076eb3-f214-448e-8647-04f8e42f52e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.245189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 98/198] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:15 2022] POST /identity/v3/users => generated 297 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:16.246136 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350caaa1-dc96-43fd-8729-7a6dff694a1f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:16.347052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f076eb3-f214-448e-8647-04f8e42f52e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:16.350549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f076eb3-f214-448e-8647-04f8e42f52e0 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:16.364629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350caaa1-dc96-43fd-8729-7a6dff694a1f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:16.406561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128f9d2-6ffb-48cf-9a28-50b439b23815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:16.407468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128f9d2-6ffb-48cf-9a28-50b439b23815 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.407810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128f9d2-6ffb-48cf-9a28-50b439b23815 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.408138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128f9d2-6ffb-48cf-9a28-50b439b23815 None admin] PATH_INFO: `/v3/users/f8c83e570c724b1f97ec66b5d5233ecb` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.409318 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128f9d2-6ffb-48cf-9a28-50b439b23815 None admin] RBAC: Authorizing `identity:get_user(user_id=f8c83e570c724b1f97ec66b5d5233ecb)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:16.411638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128f9d2-6ffb-48cf-9a28-50b439b23815 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:16.413374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 99/199] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:16 2022] GET /identity/v3/users/f8c83e570c724b1f97ec66b5d5233ecb => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:16.414248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128f9d2-6ffb-48cf-9a28-50b439b23815 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:16.436320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:16.512855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 101/200] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:16.521391 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f076eb3-f214-448e-8647-04f8e42f52e0 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:16.537531 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:16.553778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128f9d2-6ffb-48cf-9a28-50b439b23815 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:16.611102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5307b0-6cf2-4222-a00f-b05cf58f172a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:16.611850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5307b0-6cf2-4222-a00f-b05cf58f172a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.612145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5307b0-6cf2-4222-a00f-b05cf58f172a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.612551 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5307b0-6cf2-4222-a00f-b05cf58f172a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.656732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f076eb3-f214-448e-8647-04f8e42f52e0 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:16.709176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a8bc0-2056-42a0-b8c2-e87e71c547ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:16.709852 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a8bc0-2056-42a0-b8c2-e87e71c547ce None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:16.710154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a8bc0-2056-42a0-b8c2-e87e71c547ce None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:16.710449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a8bc0-2056-42a0-b8c2-e87e71c547ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:16.718977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5307b0-6cf2-4222-a00f-b05cf58f172a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:16.720474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5307b0-6cf2-4222-a00f-b05cf58f172a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:16.743752 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:16.805809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a8bc0-2056-42a0-b8c2-e87e71c547ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:16.807280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a8bc0-2056-42a0-b8c2-e87e71c547ce None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:16.833354 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:16.853469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 100/201] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:16.854841 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5307b0-6cf2-4222-a00f-b05cf58f172a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:16.924320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 102/202] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:16.925447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a8bc0-2056-42a0-b8c2-e87e71c547ce None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:16.980123 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5307b0-6cf2-4222-a00f-b05cf58f172a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.014354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:17.023440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82385d40-0b32-4ea1-942e-9660a85ce8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.024479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82385d40-0b32-4ea1-942e-9660a85ce8bc None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.024772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82385d40-0b32-4ea1-942e-9660a85ce8bc None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.025057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82385d40-0b32-4ea1-942e-9660a85ce8bc None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.027034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82385d40-0b32-4ea1-942e-9660a85ce8bc None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:17.029289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82385d40-0b32-4ea1-942e-9660a85ce8bc None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:17.030652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 101/203] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:16 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:17.031473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82385d40-0b32-4ea1-942e-9660a85ce8bc None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:17.050060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a8bc0-2056-42a0-b8c2-e87e71c547ce None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.065739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:17.087738 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c364d2-27c0-4642-b3b0-2c3f840dee24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.088630 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c364d2-27c0-4642-b3b0-2c3f840dee24 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.088919 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c364d2-27c0-4642-b3b0-2c3f840dee24 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.089202 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c364d2-27c0-4642-b3b0-2c3f840dee24 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.090466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c364d2-27c0-4642-b3b0-2c3f840dee24 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:17.091921 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c364d2-27c0-4642-b3b0-2c3f840dee24 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:17.093833 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 103/204] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:16 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:17.094597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c364d2-27c0-4642-b3b0-2c3f840dee24 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:17.171345 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82385d40-0b32-4ea1-942e-9660a85ce8bc None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.207978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c364d2-27c0-4642-b3b0-2c3f840dee24 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.212311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e81d35-7878-45e1-851f-cc2e9d4fe922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.213410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e81d35-7878-45e1-851f-cc2e9d4fe922 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.213964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e81d35-7878-45e1-851f-cc2e9d4fe922 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.214435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e81d35-7878-45e1-851f-cc2e9d4fe922 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.216002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e81d35-7878-45e1-851f-cc2e9d4fe922 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:17.217636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e81d35-7878-45e1-851f-cc2e9d4fe922 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:17.225186 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 102/205] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:17 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:17.226434 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e81d35-7878-45e1-851f-cc2e9d4fe922 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:17.245779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba982a-a60c-43cd-8547-ce3fc462fd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.246719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba982a-a60c-43cd-8547-ce3fc462fd6d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.247010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba982a-a60c-43cd-8547-ce3fc462fd6d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.247344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba982a-a60c-43cd-8547-ce3fc462fd6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.248366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba982a-a60c-43cd-8547-ce3fc462fd6d None admin] RBAC: Adding query filter params (role.id=b2f299f9533944ca821a102e13bd4c69, scope.project.id=f0435acd015c454f926f9318bb5503b2, user.id=1dc98c5904dd482eb49bd3a04afe6d28) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:17.248688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba982a-a60c-43cd-8547-ce3fc462fd6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:17.250628 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba982a-a60c-43cd-8547-ce3fc462fd6d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:17.266355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 104/206] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:17 2022] GET /identity/v3/role_assignments?user.id=1dc98c5904dd482eb49bd3a04afe6d28&scope.project.id=f0435acd015c454f926f9318bb5503b2&role.id=b2f299f9533944ca821a102e13bd4c69 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:17.266355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba982a-a60c-43cd-8547-ce3fc462fd6d None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:17.315721 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:17.374827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:17.460942 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e81d35-7878-45e1-851f-cc2e9d4fe922 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.474181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba982a-a60c-43cd-8547-ce3fc462fd6d None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:17.510885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b9482-47b8-40cc-9e35-00e64e0fb2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.512411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b9482-47b8-40cc-9e35-00e64e0fb2b0 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.512871 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b9482-47b8-40cc-9e35-00e64e0fb2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.513301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b9482-47b8-40cc-9e35-00e64e0fb2b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.514311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b9482-47b8-40cc-9e35-00e64e0fb2b0 None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=8cea8011767244e0a581010033a1caf4, user.id=f8c83e570c724b1f97ec66b5d5233ecb) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:17.514867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b9482-47b8-40cc-9e35-00e64e0fb2b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:17.516218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d554e02-845e-4055-918b-6e0d873d39a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:17.516659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b9482-47b8-40cc-9e35-00e64e0fb2b0 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:17.517280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d554e02-845e-4055-918b-6e0d873d39a4 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:17.517280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d554e02-845e-4055-918b-6e0d873d39a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:17.517462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d554e02-845e-4055-918b-6e0d873d39a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:17.518272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d554e02-845e-4055-918b-6e0d873d39a4 None admin] RBAC: Adding query filter params (role.id=a638e91b70624f669ffe76dac32323f5, scope.project.id=8cea8011767244e0a581010033a1caf4, user.id=f8c83e570c724b1f97ec66b5d5233ecb) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:17.520848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d554e02-845e-4055-918b-6e0d873d39a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:17.522345 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d554e02-845e-4055-918b-6e0d873d39a4 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:17.525260 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 103/207] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:17 2022] GET /identity/v3/role_assignments?user.id=f8c83e570c724b1f97ec66b5d5233ecb&scope.project.id=8cea8011767244e0a581010033a1caf4&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:17.534368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 105/208] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:17 2022] GET /identity/v3/role_assignments?user.id=f8c83e570c724b1f97ec66b5d5233ecb&scope.project.id=8cea8011767244e0a581010033a1caf4&role.id=a638e91b70624f669ffe76dac32323f5 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:18.618184 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17202aeb-6308-42d3-94d7-3fc2db0c4e6d None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:18.618744 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17202aeb-6308-42d3-94d7-3fc2db0c4e6d None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:18.618857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17202aeb-6308-42d3-94d7-3fc2db0c4e6d None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:18.620175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 104/209] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:18.625911 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2bc31-b254-42b4-ad20-25f0bacafc76 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:18.626437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2bc31-b254-42b4-ad20-25f0bacafc76 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:18.626511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2bc31-b254-42b4-ad20-25f0bacafc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:18.646395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c5a2bc31-b254-42b4-ad20-25f0bacafc76 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:18.647637 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c5a2bc31-b254-42b4-ad20-25f0bacafc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:18.729534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 106/210] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:18.746366 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35145cf-fae1-49c9-b4c0-15299e57f160 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:18.747279 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35145cf-fae1-49c9-b4c0-15299e57f160 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:18.747807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35145cf-fae1-49c9-b4c0-15299e57f160 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:18.749818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 105/211] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:18.753950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2bc31-b254-42b4-ad20-25f0bacafc76 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:18.757659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc9cf0-706d-4e77-a4d3-098796402e9a None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:18.758295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc9cf0-706d-4e77-a4d3-098796402e9a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:18.758959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc9cf0-706d-4e77-a4d3-098796402e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:18.781066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-36fc9cf0-706d-4e77-a4d3-098796402e9a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:18.783458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-36fc9cf0-706d-4e77-a4d3-098796402e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:18.808270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:18.868850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 106/212] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:18.880713 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2bc31-b254-42b4-ad20-25f0bacafc76 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:18.891856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc9cf0-706d-4e77-a4d3-098796402e9a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:18.924233 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95687efa-0802-46dc-8546-9cc3d9e0ee4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:18.925091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95687efa-0802-46dc-8546-9cc3d9e0ee4b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:18.925380 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95687efa-0802-46dc-8546-9cc3d9e0ee4b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:18.925688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95687efa-0802-46dc-8546-9cc3d9e0ee4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:18.939884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95687efa-0802-46dc-8546-9cc3d9e0ee4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:18.949208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-95687efa-0802-46dc-8546-9cc3d9e0ee4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:18.952700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:18.953681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:18.954634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:18.954634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:18.954634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:18.954634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:18.954634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:18.954634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:18.954634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:18.954634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:18.954634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:18.954634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:18.954634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:18.954634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:18.954634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 107/213] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:18.960695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95687efa-0802-46dc-8546-9cc3d9e0ee4b None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:19.022411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc9cf0-706d-4e77-a4d3-098796402e9a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:19.071297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068e178-3a44-4b93-badd-f301bb081609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:19.072580 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068e178-3a44-4b93-badd-f301bb081609 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:19.073067 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068e178-3a44-4b93-badd-f301bb081609 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:19.073654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068e178-3a44-4b93-badd-f301bb081609 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:19.074182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95687efa-0802-46dc-8546-9cc3d9e0ee4b None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:19.075540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068e178-3a44-4b93-badd-f301bb081609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:19.077489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068e178-3a44-4b93-badd-f301bb081609 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:19.079708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 107/214] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:19.086397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:19.087696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068e178-3a44-4b93-badd-f301bb081609 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:19.112860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4184a6-60b5-4ba8-b9c6-bb191c0b1278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:19.114381 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4184a6-60b5-4ba8-b9c6-bb191c0b1278 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:19.114567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4184a6-60b5-4ba8-b9c6-bb191c0b1278 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:19.114840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4184a6-60b5-4ba8-b9c6-bb191c0b1278 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:19.115610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4184a6-60b5-4ba8-b9c6-bb191c0b1278 None admin] RBAC: Adding query filter params (name=member) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:19.115924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4184a6-60b5-4ba8-b9c6-bb191c0b1278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:19.117641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4184a6-60b5-4ba8-b9c6-bb191c0b1278 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:19.128775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 108/215] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:19.141122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4184a6-60b5-4ba8-b9c6-bb191c0b1278 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:19.206544 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068e178-3a44-4b93-badd-f301bb081609 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:19.221867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:19.247311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a19673-d1b2-4a1c-941b-30c8725263d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:19.248497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a19673-d1b2-4a1c-941b-30c8725263d0 None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:19.249329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a19673-d1b2-4a1c-941b-30c8725263d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:19.249963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a19673-d1b2-4a1c-941b-30c8725263d0 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:19.251284 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a19673-d1b2-4a1c-941b-30c8725263d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:19.253096 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a19673-d1b2-4a1c-941b-30c8725263d0 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:19.263692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4184a6-60b5-4ba8-b9c6-bb191c0b1278 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:19.306271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38bd5b-6235-4c6a-9209-af4dd327f76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:19.307289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38bd5b-6235-4c6a-9209-af4dd327f76c None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:19.307641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38bd5b-6235-4c6a-9209-af4dd327f76c None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:19.307904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38bd5b-6235-4c6a-9209-af4dd327f76c None admin] PATH_INFO: `/v3/users/9b2c12ba651743729eda4429bad7652b` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:19.323612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38bd5b-6235-4c6a-9209-af4dd327f76c None admin] RBAC: Authorizing `identity:get_user(user_id=9b2c12ba651743729eda4429bad7652b)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:19.325306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38bd5b-6235-4c6a-9209-af4dd327f76c None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:19.327350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 109/216] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:19 2022] GET /identity/v3/users/9b2c12ba651743729eda4429bad7652b => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:19.333275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38bd5b-6235-4c6a-9209-af4dd327f76c None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:19.358980 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:19.378351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 108/217] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:19 2022] POST /identity/v3/users => generated 297 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:19.476446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38bd5b-6235-4c6a-9209-af4dd327f76c None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:19.522792 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872f6a5-022e-4c58-9797-0fed093fe714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:19.523553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872f6a5-022e-4c58-9797-0fed093fe714 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:19.523848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872f6a5-022e-4c58-9797-0fed093fe714 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:19.524183 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872f6a5-022e-4c58-9797-0fed093fe714 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:19.639331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872f6a5-022e-4c58-9797-0fed093fe714 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:19.641120 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872f6a5-022e-4c58-9797-0fed093fe714 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:19.674062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:19.679800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d2c5f-47b9-43c8-aac5-88d53619bd1f None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:19.679800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d2c5f-47b9-43c8-aac5-88d53619bd1f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:19.680420 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d2c5f-47b9-43c8-aac5-88d53619bd1f None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:19.685618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 109/218] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:19.695882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248564a-8d0f-4d43-8505-8c00b3d63127 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:19.696229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248564a-8d0f-4d43-8505-8c00b3d63127 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:19.696515 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248564a-8d0f-4d43-8505-8c00b3d63127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:19.718446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-e248564a-8d0f-4d43-8505-8c00b3d63127 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:19.719695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e248564a-8d0f-4d43-8505-8c00b3d63127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:19.798079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 110/219] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 18:42:19.799743 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd41016-04d7-4a7d-b415-4058ad8f97f9 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:19.800080 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd41016-04d7-4a7d-b415-4058ad8f97f9 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:19.800385 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd41016-04d7-4a7d-b415-4058ad8f97f9 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:19.802856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 111/220] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:19.808609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd41016-04d7-4a7d-b415-4058ad8f97f9 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:19.831035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 110/221] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:19.839859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521515c-2188-4696-8904-4733bdc87eba None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:19.840207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521515c-2188-4696-8904-4733bdc87eba None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:19.840432 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521515c-2188-4696-8904-4733bdc87eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:19.862209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-2521515c-2188-4696-8904-4733bdc87eba None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:19.864137 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-2521515c-2188-4696-8904-4733bdc87eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:19.950958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 111/222] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:19.959296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521515c-2188-4696-8904-4733bdc87eba None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:19.981703 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd41016-04d7-4a7d-b415-4058ad8f97f9 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:19.999415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:20.023213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:20.045557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d618b3-e307-48e8-aff2-889b91d516d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.047116 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d618b3-e307-48e8-aff2-889b91d516d2 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.047825 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d618b3-e307-48e8-aff2-889b91d516d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.048442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d618b3-e307-48e8-aff2-889b91d516d2 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.050361 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d618b3-e307-48e8-aff2-889b91d516d2 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:20.053029 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d618b3-e307-48e8-aff2-889b91d516d2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:20.055886 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 112/223] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:19 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:20.057148 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d618b3-e307-48e8-aff2-889b91d516d2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.103896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521515c-2188-4696-8904-4733bdc87eba None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.165349 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782aad4-3c69-41f4-8432-a109291ee7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.166199 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782aad4-3c69-41f4-8432-a109291ee7fb None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.166520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782aad4-3c69-41f4-8432-a109291ee7fb None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.166840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782aad4-3c69-41f4-8432-a109291ee7fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.187206 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782aad4-3c69-41f4-8432-a109291ee7fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-f782aad4-3c69-41f4-8432-a109291ee7fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:20.199240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:20.200189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:20.200973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:20.200973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:20.200973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:20.200973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:20.200973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:20.200973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:20.200973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:20.200973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:20.200973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:20.200973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:20.200973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:20.200973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:20.204229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 112/224] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:20.205558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782aad4-3c69-41f4-8432-a109291ee7fb None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.216349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d618b3-e307-48e8-aff2-889b91d516d2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.260171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f772b71-b766-4626-ba8d-7c6db42271aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.261164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f772b71-b766-4626-ba8d-7c6db42271aa None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.261522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f772b71-b766-4626-ba8d-7c6db42271aa None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.261914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f772b71-b766-4626-ba8d-7c6db42271aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.276439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f772b71-b766-4626-ba8d-7c6db42271aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-9f772b71-b766-4626-ba8d-7c6db42271aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:20.285577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:20.288759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 113/225] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:20.289642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f772b71-b766-4626-ba8d-7c6db42271aa None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.346110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:20.362616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782aad4-3c69-41f4-8432-a109291ee7fb None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.363649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:20.414540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fa143-f04b-4a8f-a9bd-d38bde54b13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.415905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fa143-f04b-4a8f-a9bd-d38bde54b13a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.416534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fa143-f04b-4a8f-a9bd-d38bde54b13a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.417022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fa143-f04b-4a8f-a9bd-d38bde54b13a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.418686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fa143-f04b-4a8f-a9bd-d38bde54b13a None admin] RBAC: Adding query filter params (role.id=9dc988eb162e47e48df810053eecb165, scope.project.id=8cea8011767244e0a581010033a1caf4, user.id=9b2c12ba651743729eda4429bad7652b) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:20.419371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fa143-f04b-4a8f-a9bd-d38bde54b13a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:20.421986 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fa143-f04b-4a8f-a9bd-d38bde54b13a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:20.431870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 113/226] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/role_assignments?user.id=9b2c12ba651743729eda4429bad7652b&scope.project.id=8cea8011767244e0a581010033a1caf4&role.id=9dc988eb162e47e48df810053eecb165 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:20.433060 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fa143-f04b-4a8f-a9bd-d38bde54b13a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.440783 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f772b71-b766-4626-ba8d-7c6db42271aa None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.532434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2743e-6c70-4eee-85c1-8781188665e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.533525 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2743e-6c70-4eee-85c1-8781188665e1 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.534058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2743e-6c70-4eee-85c1-8781188665e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.534595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2743e-6c70-4eee-85c1-8781188665e1 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.536717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2743e-6c70-4eee-85c1-8781188665e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:20.537340 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2743e-6c70-4eee-85c1-8781188665e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:20.540335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2743e-6c70-4eee-85c1-8781188665e1 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:20.563353 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 114/227] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:20.571753 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2743e-6c70-4eee-85c1-8781188665e1 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.662927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fa143-f04b-4a8f-a9bd-d38bde54b13a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.702592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46244fe-f914-4575-95f4-1c392800eb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.703235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46244fe-f914-4575-95f4-1c392800eb9a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.703479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46244fe-f914-4575-95f4-1c392800eb9a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.703755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46244fe-f914-4575-95f4-1c392800eb9a None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.704517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46244fe-f914-4575-95f4-1c392800eb9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:20.704841 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46244fe-f914-4575-95f4-1c392800eb9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:20.706222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46244fe-f914-4575-95f4-1c392800eb9a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:20.720748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 114/228] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:20.722380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46244fe-f914-4575-95f4-1c392800eb9a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.725927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2743e-6c70-4eee-85c1-8781188665e1 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.738043 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:20.741322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:20.776220 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8236c-d196-4837-9d3f-a490ed774d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.777154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8236c-d196-4837-9d3f-a490ed774d55 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.777441 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8236c-d196-4837-9d3f-a490ed774d55 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.777723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8236c-d196-4837-9d3f-a490ed774d55 None admin] PATH_INFO: `/v3/users/f8c83e570c724b1f97ec66b5d5233ecb` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.779138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8236c-d196-4837-9d3f-a490ed774d55 None admin] RBAC: Authorizing `identity:get_user(user_id=f8c83e570c724b1f97ec66b5d5233ecb)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:20.780622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8236c-d196-4837-9d3f-a490ed774d55 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:20.787674 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 115/229] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/users/f8c83e570c724b1f97ec66b5d5233ecb => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:20.789037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8236c-d196-4837-9d3f-a490ed774d55 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:20.860415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46244fe-f914-4575-95f4-1c392800eb9a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:20.899861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26941f0e-78da-4760-ad42-d7ec878c1a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:20.900755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26941f0e-78da-4760-ad42-d7ec878c1a32 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:20.901044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26941f0e-78da-4760-ad42-d7ec878c1a32 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:20.901464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26941f0e-78da-4760-ad42-d7ec878c1a32 None admin] PATH_INFO: `/v3/users/f8c83e570c724b1f97ec66b5d5233ecb` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:20.902754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26941f0e-78da-4760-ad42-d7ec878c1a32 None admin] RBAC: Authorizing `identity:get_user(user_id=f8c83e570c724b1f97ec66b5d5233ecb)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:20.904229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26941f0e-78da-4760-ad42-d7ec878c1a32 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:20.911035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 115/230] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/users/f8c83e570c724b1f97ec66b5d5233ecb => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:20.912443 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26941f0e-78da-4760-ad42-d7ec878c1a32 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:21.026533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:21.046612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26941f0e-78da-4760-ad42-d7ec878c1a32 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:21.067695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8236c-d196-4837-9d3f-a490ed774d55 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:21.135893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ecf70-a808-4a34-9ce8-6ea5eb05f8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:21.137559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ecf70-a808-4a34-9ce8-6ea5eb05f8b2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:21.137856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ecf70-a808-4a34-9ce8-6ea5eb05f8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:21.138142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ecf70-a808-4a34-9ce8-6ea5eb05f8b2 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:21.139900 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ecf70-a808-4a34-9ce8-6ea5eb05f8b2 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:21.141373 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ecf70-a808-4a34-9ce8-6ea5eb05f8b2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:21.144990 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0be961-12a4-4b87-ac07-31495f044494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:21.146060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0be961-12a4-4b87-ac07-31495f044494 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:21.146368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0be961-12a4-4b87-ac07-31495f044494 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:21.146661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0be961-12a4-4b87-ac07-31495f044494 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:21.148312 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0be961-12a4-4b87-ac07-31495f044494 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:21.149846 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0be961-12a4-4b87-ac07-31495f044494 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:21.151672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 116/231] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:21.152439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0be961-12a4-4b87-ac07-31495f044494 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:21.152987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 116/232] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:20 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:21.158962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ecf70-a808-4a34-9ce8-6ea5eb05f8b2 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:21.259126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:21.359829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0be961-12a4-4b87-ac07-31495f044494 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:21.384885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ecf70-a808-4a34-9ce8-6ea5eb05f8b2 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:21.399014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb841c79-eb8e-4172-91cb-0f2912bedeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:21.399944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb841c79-eb8e-4172-91cb-0f2912bedeea None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:21.400236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb841c79-eb8e-4172-91cb-0f2912bedeea None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:21.400522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb841c79-eb8e-4172-91cb-0f2912bedeea None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4/users/f8c83e570c724b1f97ec66b5d5233ecb/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:21.402824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb841c79-eb8e-4172-91cb-0f2912bedeea None admin] RBAC: Authorizing `identity:create_grant(project_id=8cea8011767244e0a581010033a1caf4, user_id=f8c83e570c724b1f97ec66b5d5233ecb, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:21.404385 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb841c79-eb8e-4172-91cb-0f2912bedeea None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:21.445541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c0006-db52-4eee-9b81-e509ea55f4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:21.446738 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c0006-db52-4eee-9b81-e509ea55f4f5 None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:21.447268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c0006-db52-4eee-9b81-e509ea55f4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:21.447602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c0006-db52-4eee-9b81-e509ea55f4f5 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4/users/f8c83e570c724b1f97ec66b5d5233ecb/roles/a638e91b70624f669ffe76dac32323f5` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:21.450695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c0006-db52-4eee-9b81-e509ea55f4f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8cea8011767244e0a581010033a1caf4, user_id=f8c83e570c724b1f97ec66b5d5233ecb, role_id=a638e91b70624f669ffe76dac32323f5)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:21.452433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c0006-db52-4eee-9b81-e509ea55f4f5 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:21.497207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 117/233] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:21 2022] PUT /identity/v3/projects/8cea8011767244e0a581010033a1caf4/users/f8c83e570c724b1f97ec66b5d5233ecb/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:21.562700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 117/234] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:21 2022] PUT /identity/v3/projects/8cea8011767244e0a581010033a1caf4/users/f8c83e570c724b1f97ec66b5d5233ecb/roles/a638e91b70624f669ffe76dac32323f5 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 18:42:22.023497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9a0a4-882b-4591-ba36-8eb3b9fa7a5a None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:22.024009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9a0a4-882b-4591-ba36-8eb3b9fa7a5a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:22.024115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9a0a4-882b-4591-ba36-8eb3b9fa7a5a None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:22.025453 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 118/235] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:22.031864 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a7118-c64b-4122-b1d2-acb6cf43018c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:22.032338 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a7118-c64b-4122-b1d2-acb6cf43018c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:22.032412 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a7118-c64b-4122-b1d2-acb6cf43018c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:22.059439 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-009a7118-c64b-4122-b1d2-acb6cf43018c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:22.060665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-009a7118-c64b-4122-b1d2-acb6cf43018c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:22.063958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:22.160706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 118/236] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:22.202857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9a0a4-882b-4591-ba36-8eb3b9fa7a5a None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:22.324418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc25fcb-eaaa-49d1-9967-2d76ffcdcdad None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:22.325128 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc25fcb-eaaa-49d1-9967-2d76ffcdcdad None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:22.325622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc25fcb-eaaa-49d1-9967-2d76ffcdcdad None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:22.327266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 119/237] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:22.333302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bc4e9-4950-4516-b949-8d660066b960 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:22.334140 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bc4e9-4950-4516-b949-8d660066b960 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:22.334669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bc4e9-4950-4516-b949-8d660066b960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:22.340157 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9a0a4-882b-4591-ba36-8eb3b9fa7a5a None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:22.357494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:22.370058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-e18bc4e9-4950-4516-b949-8d660066b960 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:22.373122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e18bc4e9-4950-4516-b949-8d660066b960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:22.401783 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f1cda-027c-4ded-9a88-49b66e5db150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:22.404156 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f1cda-027c-4ded-9a88-49b66e5db150 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:22.404454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f1cda-027c-4ded-9a88-49b66e5db150 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:22.404737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f1cda-027c-4ded-9a88-49b66e5db150 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:22.426005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f1cda-027c-4ded-9a88-49b66e5db150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-807f1cda-027c-4ded-9a88-49b66e5db150 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:22.435654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:22.437319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:22.438678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:22.438678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:22.438678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:22.438678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:22.438678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:22.438678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:22.438678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:22.438678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:22.438678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:22.438678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:22.438678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:22.438678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:22.438678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 119/238] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:22.445144 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f1cda-027c-4ded-9a88-49b66e5db150 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:22.517567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 120/239] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:22.560823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bc4e9-4950-4516-b949-8d660066b960 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:22.611568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f1cda-027c-4ded-9a88-49b66e5db150 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:22.668125 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7222c-eef0-435c-9a64-7157f0565691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:22.669186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7222c-eef0-435c-9a64-7157f0565691 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:22.669723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7222c-eef0-435c-9a64-7157f0565691 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:22.670199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7222c-eef0-435c-9a64-7157f0565691 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:22.672512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7222c-eef0-435c-9a64-7157f0565691 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:22.673040 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7222c-eef0-435c-9a64-7157f0565691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:22.674720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7222c-eef0-435c-9a64-7157f0565691 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:22.694718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 120/240] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:22.700308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7222c-eef0-435c-9a64-7157f0565691 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:22.735967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:22.752028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:22.820059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bc4e9-4950-4516-b949-8d660066b960 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:22.884728 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7222c-eef0-435c-9a64-7157f0565691 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:22.905291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ffe65-aa0f-4a82-bb8a-1684e24b6c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:22.906627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ffe65-aa0f-4a82-bb8a-1684e24b6c18 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:22.907117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ffe65-aa0f-4a82-bb8a-1684e24b6c18 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:22.907680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ffe65-aa0f-4a82-bb8a-1684e24b6c18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:22.913552 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ffe65-aa0f-4a82-bb8a-1684e24b6c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:22.915471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ffe65-aa0f-4a82-bb8a-1684e24b6c18 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:22.917695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 121/241] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:22.923660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ffe65-aa0f-4a82-bb8a-1684e24b6c18 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:22.938377 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe8f3f-06a1-430f-b317-fb36e5f81aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:22.939390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe8f3f-06a1-430f-b317-fb36e5f81aad None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:22.939710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe8f3f-06a1-430f-b317-fb36e5f81aad None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:22.939987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe8f3f-06a1-430f-b317-fb36e5f81aad None admin] PATH_INFO: `/v3/users/09adf329fc2a484092b29e52cf4c6c0f` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:22.987184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe8f3f-06a1-430f-b317-fb36e5f81aad None admin] RBAC: Authorizing `identity:get_user(user_id=09adf329fc2a484092b29e52cf4c6c0f)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:22.988728 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe8f3f-06a1-430f-b317-fb36e5f81aad None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:22.991076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 121/242] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:22 2022] GET /identity/v3/users/09adf329fc2a484092b29e52cf4c6c0f => generated 297 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:22.999979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe8f3f-06a1-430f-b317-fb36e5f81aad None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:23.077657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ffe65-aa0f-4a82-bb8a-1684e24b6c18 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:23.109595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:23.125484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e803a4-b031-4ebe-aed9-f85c808a06db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:23.126589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e803a4-b031-4ebe-aed9-f85c808a06db None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:23.127095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e803a4-b031-4ebe-aed9-f85c808a06db None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:23.127545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e803a4-b031-4ebe-aed9-f85c808a06db None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:23.128571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e803a4-b031-4ebe-aed9-f85c808a06db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:23.128696 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe8f3f-06a1-430f-b317-fb36e5f81aad None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:23.131445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e803a4-b031-4ebe-aed9-f85c808a06db None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:23.144414 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:23.181764 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba621731-c35c-4cd3-bb34-2195077e7c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:23.182557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba621731-c35c-4cd3-bb34-2195077e7c16 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:23.182845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba621731-c35c-4cd3-bb34-2195077e7c16 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:23.183125 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba621731-c35c-4cd3-bb34-2195077e7c16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:23.234366 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 122/243] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:22 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:42:23.234366 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0063b-843f-4296-bd76-258f4f9cbc4d None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:23.234366 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0063b-843f-4296-bd76-258f4f9cbc4d None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:23.234366 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0063b-843f-4296-bd76-258f4f9cbc4d None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:23.234366 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 123/244] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:23.244392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faa10f-4d03-4cbe-a6d1-b2915ab6acd9 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:23.247628 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faa10f-4d03-4cbe-a6d1-b2915ab6acd9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:23.247960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faa10f-4d03-4cbe-a6d1-b2915ab6acd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:23.286242 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-12faa10f-4d03-4cbe-a6d1-b2915ab6acd9 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:23.287856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-12faa10f-4d03-4cbe-a6d1-b2915ab6acd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:23.341317 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba621731-c35c-4cd3-bb34-2195077e7c16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:23.342883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba621731-c35c-4cd3-bb34-2195077e7c16 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:23.463468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 124/245] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:23.491000 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faa10f-4d03-4cbe-a6d1-b2915ab6acd9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:23.496603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:23.518062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 122/246] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:23.525254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba621731-c35c-4cd3-bb34-2195077e7c16 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:23.663720 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faa10f-4d03-4cbe-a6d1-b2915ab6acd9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:23.680206 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba621731-c35c-4cd3-bb34-2195077e7c16 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:23.702429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:23.724473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c19330-eb41-4eaa-9554-d3183efdd416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:23.725459 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c19330-eb41-4eaa-9554-d3183efdd416 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:23.725858 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c19330-eb41-4eaa-9554-d3183efdd416 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:23.726261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c19330-eb41-4eaa-9554-d3183efdd416 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:23.744552 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c19330-eb41-4eaa-9554-d3183efdd416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:23.753356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e212de4-2a2c-4a87-bd76-c81d74df6f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:23.755262 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e212de4-2a2c-4a87-bd76-c81d74df6f54 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-a6c19330-eb41-4eaa-9554-d3183efdd416 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:23.755530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:23.756489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:23.757354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:23.757354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:23.757354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:23.757354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:23.757354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:23.757354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:23.757354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:23.757354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:23.757354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:23.757354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:23.757354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:23.757354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:23.757959 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e212de4-2a2c-4a87-bd76-c81d74df6f54 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:23.757959 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e212de4-2a2c-4a87-bd76-c81d74df6f54 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:23.758107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 125/247] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:23.759125 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e212de4-2a2c-4a87-bd76-c81d74df6f54 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:23.760607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e212de4-2a2c-4a87-bd76-c81d74df6f54 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:23.762878 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 123/248] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:23 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:23.767019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c19330-eb41-4eaa-9554-d3183efdd416 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:23.768995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e212de4-2a2c-4a87-bd76-c81d74df6f54 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:23.848159 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:23.914686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c19330-eb41-4eaa-9554-d3183efdd416 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:23.917981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e212de4-2a2c-4a87-bd76-c81d74df6f54 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:23.962752 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebefddd-70ff-492a-ab94-048fc8f89fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:23.963781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebefddd-70ff-492a-ab94-048fc8f89fab None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:23.964076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebefddd-70ff-492a-ab94-048fc8f89fab None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:23.964884 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebefddd-70ff-492a-ab94-048fc8f89fab None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:23.967639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebefddd-70ff-492a-ab94-048fc8f89fab None admin] RBAC: Adding query filter params (name=member) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:23.967962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebefddd-70ff-492a-ab94-048fc8f89fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:23.969480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebefddd-70ff-492a-ab94-048fc8f89fab None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:23.975166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395b4f9-cda3-4c61-8bb1-7fed74397497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:23.975793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395b4f9-cda3-4c61-8bb1-7fed74397497 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:23.976087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395b4f9-cda3-4c61-8bb1-7fed74397497 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:23.976363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395b4f9-cda3-4c61-8bb1-7fed74397497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:23.977213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395b4f9-cda3-4c61-8bb1-7fed74397497 None admin] RBAC: Adding query filter params (role.id=b2f299f9533944ca821a102e13bd4c69, scope.project.id=8cea8011767244e0a581010033a1caf4, user.id=09adf329fc2a484092b29e52cf4c6c0f) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:23.977531 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395b4f9-cda3-4c61-8bb1-7fed74397497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:23.979036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395b4f9-cda3-4c61-8bb1-7fed74397497 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:23.980429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 126/249] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:23.987814 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebefddd-70ff-492a-ab94-048fc8f89fab None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:23.990462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 124/250] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:23 2022] GET /identity/v3/role_assignments?user.id=09adf329fc2a484092b29e52cf4c6c0f&scope.project.id=8cea8011767244e0a581010033a1caf4&role.id=b2f299f9533944ca821a102e13bd4c69 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:24.037885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898ca22-e143-4e37-bd14-b3dd9157be84 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:24.038602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898ca22-e143-4e37-bd14-b3dd9157be84 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:24.039103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898ca22-e143-4e37-bd14-b3dd9157be84 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:24.040924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 125/251] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:24.047418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad200c3-fb81-4817-9b7d-a9691eb2235b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:24.048128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad200c3-fb81-4817-9b7d-a9691eb2235b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:24.048634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad200c3-fb81-4817-9b7d-a9691eb2235b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:24.055024 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:24.075635 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-aad200c3-fb81-4817-9b7d-a9691eb2235b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:24.082022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-aad200c3-fb81-4817-9b7d-a9691eb2235b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:24.122128 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebefddd-70ff-492a-ab94-048fc8f89fab None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:24.217418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19f90d-b79d-4c30-be67-6d10f72e46df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:24.222356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19f90d-b79d-4c30-be67-6d10f72e46df None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:24.222356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19f90d-b79d-4c30-be67-6d10f72e46df None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:24.222356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19f90d-b79d-4c30-be67-6d10f72e46df None admin] PATH_INFO: `/v3/users/9b2c12ba651743729eda4429bad7652b` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:24.222356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19f90d-b79d-4c30-be67-6d10f72e46df None admin] RBAC: Authorizing `identity:get_user(user_id=9b2c12ba651743729eda4429bad7652b)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:24.223556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19f90d-b79d-4c30-be67-6d10f72e46df None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:24.225966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 127/252] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:23 2022] GET /identity/v3/users/9b2c12ba651743729eda4429bad7652b => generated 297 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:24.236444 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19f90d-b79d-4c30-be67-6d10f72e46df None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:24.249024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 126/253] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:24.296334 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad200c3-fb81-4817-9b7d-a9691eb2235b None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:24.407275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:24.498367 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19f90d-b79d-4c30-be67-6d10f72e46df None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:24.513524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad200c3-fb81-4817-9b7d-a9691eb2235b None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:24.531516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:24.563086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c153030-b5eb-4769-99ee-8ebebf6f4e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:24.563954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c153030-b5eb-4769-99ee-8ebebf6f4e45 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:24.564285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c153030-b5eb-4769-99ee-8ebebf6f4e45 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:24.564573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c153030-b5eb-4769-99ee-8ebebf6f4e45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:24.565598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d7402-71df-48d1-9ff5-5b35ee474d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:24.566541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d7402-71df-48d1-9ff5-5b35ee474d82 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:24.566884 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d7402-71df-48d1-9ff5-5b35ee474d82 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:24.567932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d7402-71df-48d1-9ff5-5b35ee474d82 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:24.571114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d7402-71df-48d1-9ff5-5b35ee474d82 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:24.572613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d7402-71df-48d1-9ff5-5b35ee474d82 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:24.574426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 128/254] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:24 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:24.575834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee802c33-bbd0-4f99-b29d-cde79d165b35 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:24.576127 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee802c33-bbd0-4f99-b29d-cde79d165b35 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:24.576406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee802c33-bbd0-4f99-b29d-cde79d165b35 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:24.577674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 129/255] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:24.588840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee802c33-bbd0-4f99-b29d-cde79d165b35 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:24.597509 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c153030-b5eb-4769-99ee-8ebebf6f4e45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-5c153030-b5eb-4769-99ee-8ebebf6f4e45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:24.605494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:24.606507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:24.607379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:24.607379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:24.607379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:24.607379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:24.607379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:24.607379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:24.607379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:24.607379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:24.607379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:24.607379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:24.607379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:24.607379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:24.616431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 127/256] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:24.617840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16fca5-967d-4cf8-b79b-4cc27744c905 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:24.618137 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16fca5-967d-4cf8-b79b-4cc27744c905 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:24.622577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16fca5-967d-4cf8-b79b-4cc27744c905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:24.666261 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-4e16fca5-967d-4cf8-b79b-4cc27744c905 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:24.671395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4e16fca5-967d-4cf8-b79b-4cc27744c905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:24.817493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee802c33-bbd0-4f99-b29d-cde79d165b35 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:24.836151 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 128/257] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:24.843597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16fca5-967d-4cf8-b79b-4cc27744c905 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:24.904964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08991756-331a-49ac-bbe1-9ca2904e6d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:24.906376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08991756-331a-49ac-bbe1-9ca2904e6d3e None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:24.906905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08991756-331a-49ac-bbe1-9ca2904e6d3e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:24.907376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08991756-331a-49ac-bbe1-9ca2904e6d3e None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4/users/9b2c12ba651743729eda4429bad7652b/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:24.910303 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08991756-331a-49ac-bbe1-9ca2904e6d3e None admin] RBAC: Authorizing `identity:create_grant(project_id=8cea8011767244e0a581010033a1caf4, user_id=9b2c12ba651743729eda4429bad7652b, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:24.912703 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08991756-331a-49ac-bbe1-9ca2904e6d3e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:24.939950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:24.999563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16fca5-967d-4cf8-b79b-4cc27744c905 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:25.031050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 130/258] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:24 2022] PUT /identity/v3/projects/8cea8011767244e0a581010033a1caf4/users/9b2c12ba651743729eda4429bad7652b/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:25.038367 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08991756-331a-49ac-bbe1-9ca2904e6d3e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:25.042467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff1004-3801-4065-94e3-14ce000dd2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:25.046231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff1004-3801-4065-94e3-14ce000dd2c3 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:25.046597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff1004-3801-4065-94e3-14ce000dd2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:25.046891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff1004-3801-4065-94e3-14ce000dd2c3 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:25.047704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff1004-3801-4065-94e3-14ce000dd2c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:25.048086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff1004-3801-4065-94e3-14ce000dd2c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:25.049521 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff1004-3801-4065-94e3-14ce000dd2c3 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:25.068218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 129/259] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:25.074082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff1004-3801-4065-94e3-14ce000dd2c3 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:25.147661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:25.260800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08991756-331a-49ac-bbe1-9ca2904e6d3e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:25.316641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6b539-4647-4add-9157-9a1f975ee070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:25.317515 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6b539-4647-4add-9157-9a1f975ee070 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:25.317801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6b539-4647-4add-9157-9a1f975ee070 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:25.318159 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6b539-4647-4add-9157-9a1f975ee070 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:25.345143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6b539-4647-4add-9157-9a1f975ee070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-d2a6b539-4647-4add-9157-9a1f975ee070 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:25.362747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:25.364019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:25.364846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:25.364846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:25.364846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:25.364846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:25.364846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:25.364846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:25.364846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:25.364846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:25.364846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:25.364846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:25.364846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:25.364846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:25.364846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 131/260] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:25.368106 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff1004-3801-4065-94e3-14ce000dd2c3 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:25.373947 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6b539-4647-4add-9157-9a1f975ee070 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:25.383432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:25.448133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d5b45-993e-4168-b31e-2804ced08514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:25.453996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d5b45-993e-4168-b31e-2804ced08514 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:25.454290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d5b45-993e-4168-b31e-2804ced08514 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:25.454600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d5b45-993e-4168-b31e-2804ced08514 None admin] PATH_INFO: `/v3/users/f8c83e570c724b1f97ec66b5d5233ecb` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:25.458614 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d5b45-993e-4168-b31e-2804ced08514 None admin] RBAC: Authorizing `identity:get_user(user_id=f8c83e570c724b1f97ec66b5d5233ecb)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:25.460108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d5b45-993e-4168-b31e-2804ced08514 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:25.464097 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 130/261] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:25 2022] GET /identity/v3/users/f8c83e570c724b1f97ec66b5d5233ecb => generated 283 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:25.471406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d5b45-993e-4168-b31e-2804ced08514 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:25.608889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6b539-4647-4add-9157-9a1f975ee070 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:25.611282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:25.691092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cf599-23f4-4d09-93b3-cf5edabc36c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:25.691798 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cf599-23f4-4d09-93b3-cf5edabc36c1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:25.692124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cf599-23f4-4d09-93b3-cf5edabc36c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:25.692414 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cf599-23f4-4d09-93b3-cf5edabc36c1 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:25.693186 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cf599-23f4-4d09-93b3-cf5edabc36c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:25.693507 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cf599-23f4-4d09-93b3-cf5edabc36c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:25.694946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cf599-23f4-4d09-93b3-cf5edabc36c1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:25.714203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 132/262] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:25.724088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cf599-23f4-4d09-93b3-cf5edabc36c1 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:25.757006 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d5b45-993e-4168-b31e-2804ced08514 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:25.843071 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c509eed-943e-414d-b6c6-5225456ad097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:25.843759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c509eed-943e-414d-b6c6-5225456ad097 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:25.844053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c509eed-943e-414d-b6c6-5225456ad097 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:25.844337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c509eed-943e-414d-b6c6-5225456ad097 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:25.902788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:25.962363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c509eed-943e-414d-b6c6-5225456ad097 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:25.963980 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c509eed-943e-414d-b6c6-5225456ad097 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:26.031952 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cf599-23f4-4d09-93b3-cf5edabc36c1 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:26.152634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363acb3-d349-4726-8823-cfa17202efd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:26.153598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363acb3-d349-4726-8823-cfa17202efd2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.153836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363acb3-d349-4726-8823-cfa17202efd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.154582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363acb3-d349-4726-8823-cfa17202efd2 None admin] PATH_INFO: `/v3/users/f8c83e570c724b1f97ec66b5d5233ecb` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.159184 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3363acb3-d349-4726-8823-cfa17202efd2 None admin] RBAC: Authorizing `identity:get_user(user_id=f8c83e570c724b1f97ec66b5d5233ecb)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:26.161231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3363acb3-d349-4726-8823-cfa17202efd2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:26.163360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 133/263] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:25 2022] GET /identity/v3/users/f8c83e570c724b1f97ec66b5d5233ecb => generated 283 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:26.168707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363acb3-d349-4726-8823-cfa17202efd2 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:26.174416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 131/264] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:26.184559 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c509eed-943e-414d-b6c6-5225456ad097 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:26.250682 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:26.303624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:26.339247 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c509eed-943e-414d-b6c6-5225456ad097 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:26.339877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363acb3-d349-4726-8823-cfa17202efd2 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:26.384787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bf0fa-99e1-4bb8-a57d-5cf70ab4d964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:26.385681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bf0fa-99e1-4bb8-a57d-5cf70ab4d964 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.385969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bf0fa-99e1-4bb8-a57d-5cf70ab4d964 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.386247 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bf0fa-99e1-4bb8-a57d-5cf70ab4d964 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.389872 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9bf0fa-99e1-4bb8-a57d-5cf70ab4d964 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:26.391394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9bf0fa-99e1-4bb8-a57d-5cf70ab4d964 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:26.393186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 132/265] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:26 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:26.401665 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bf0fa-99e1-4bb8-a57d-5cf70ab4d964 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:26.446957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b728f23-8afa-4684-afa3-7ce9f1d4d914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:26.447695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b728f23-8afa-4684-afa3-7ce9f1d4d914 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.447991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b728f23-8afa-4684-afa3-7ce9f1d4d914 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.448277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b728f23-8afa-4684-afa3-7ce9f1d4d914 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.592407 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b728f23-8afa-4684-afa3-7ce9f1d4d914 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:26.594031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b728f23-8afa-4684-afa3-7ce9f1d4d914 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:26.612541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bf0fa-99e1-4bb8-a57d-5cf70ab4d964 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:26.629046 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:26.651807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d153f1a-73af-45fa-918a-1449b5203f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:26.652639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d153f1a-73af-45fa-918a-1449b5203f0d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.653036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d153f1a-73af-45fa-918a-1449b5203f0d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.653426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d153f1a-73af-45fa-918a-1449b5203f0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.654403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d153f1a-73af-45fa-918a-1449b5203f0d None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=8cea8011767244e0a581010033a1caf4, user.id=f8c83e570c724b1f97ec66b5d5233ecb) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:26.654971 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d153f1a-73af-45fa-918a-1449b5203f0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:26.656402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d153f1a-73af-45fa-918a-1449b5203f0d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:26.663712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 133/266] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:26 2022] GET /identity/v3/role_assignments?user.id=f8c83e570c724b1f97ec66b5d5233ecb&scope.project.id=8cea8011767244e0a581010033a1caf4&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:26.665078 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ce4bc-460e-4e0c-95e4-1f7d6660d8e6 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.665605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ce4bc-460e-4e0c-95e4-1f7d6660d8e6 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.666145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ce4bc-460e-4e0c-95e4-1f7d6660d8e6 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.667660 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 134/267] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:26.669027 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ce4e5-017b-4b35-a094-985b73806c75 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.669532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ce4e5-017b-4b35-a094-985b73806c75 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.670026 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ce4e5-017b-4b35-a094-985b73806c75 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.671610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 135/268] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:26.675982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7451a-0c08-4f09-9990-7e7ed593a6c6 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.677691 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7451a-0c08-4f09-9990-7e7ed593a6c6 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.678203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7451a-0c08-4f09-9990-7e7ed593a6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.728690 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-2df7451a-0c08-4f09-9990-7e7ed593a6c6 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:26.730519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2df7451a-0c08-4f09-9990-7e7ed593a6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:26.752861 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:26.763568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 134/269] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:26.765349 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139838b8-3aba-485f-b09f-eb6712a21ba3 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.766401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139838b8-3aba-485f-b09f-eb6712a21ba3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.766909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139838b8-3aba-485f-b09f-eb6712a21ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.790399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-139838b8-3aba-485f-b09f-eb6712a21ba3 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:26.793144 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-139838b8-3aba-485f-b09f-eb6712a21ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:26.831224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 136/270] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:26.833333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e90d8-8034-4037-a6c2-fcf2ffc45032 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.833689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e90d8-8034-4037-a6c2-fcf2ffc45032 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.834160 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e90d8-8034-4037-a6c2-fcf2ffc45032 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.836183 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 137/271] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:26.838422 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e90d8-8034-4037-a6c2-fcf2ffc45032 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:26.901988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 135/272] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:26.905324 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef52cea-a722-4c56-8341-f30f2da5b96d None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:26.905324 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef52cea-a722-4c56-8341-f30f2da5b96d None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:26.905324 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef52cea-a722-4c56-8341-f30f2da5b96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:26.931320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-3ef52cea-a722-4c56-8341-f30f2da5b96d None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:26.931662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3ef52cea-a722-4c56-8341-f30f2da5b96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:26.983488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e90d8-8034-4037-a6c2-fcf2ffc45032 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.013464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.027452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.029590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 136/273] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:27.032401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef52cea-a722-4c56-8341-f30f2da5b96d None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.042788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef289e6-d4c7-4511-8008-a5aed194c336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.043691 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef289e6-d4c7-4511-8008-a5aed194c336 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.043981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef289e6-d4c7-4511-8008-a5aed194c336 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.044265 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef289e6-d4c7-4511-8008-a5aed194c336 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.047153 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef289e6-d4c7-4511-8008-a5aed194c336 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.048735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef289e6-d4c7-4511-8008-a5aed194c336 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.050703 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 138/274] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:26 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:27.051641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef289e6-d4c7-4511-8008-a5aed194c336 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.170838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef52cea-a722-4c56-8341-f30f2da5b96d None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.177393 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef289e6-d4c7-4511-8008-a5aed194c336 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.213605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4de0ef-9b72-4311-9ec2-edb2d757588a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.214543 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de0ef-9b72-4311-9ec2-edb2d757588a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.214843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de0ef-9b72-4311-9ec2-edb2d757588a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.215115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de0ef-9b72-4311-9ec2-edb2d757588a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.225328 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3a40f-9c4f-4d43-b600-905982abe64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.226737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3a40f-9c4f-4d43-b600-905982abe64f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.226986 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4de0ef-9b72-4311-9ec2-edb2d757588a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:27.227100 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3a40f-9c4f-4d43-b600-905982abe64f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.227327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3a40f-9c4f-4d43-b600-905982abe64f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.228775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3a40f-9c4f-4d43-b600-905982abe64f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.230092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3a40f-9c4f-4d43-b600-905982abe64f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.237810 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 139/275] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:27.237810 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3a40f-9c4f-4d43-b600-905982abe64f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-4a4de0ef-9b72-4311-9ec2-edb2d757588a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:27.238064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:27.239438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 137/276] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:27.240376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4de0ef-9b72-4311-9ec2-edb2d757588a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.307374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.319319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.359033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3a40f-9c4f-4d43-b600-905982abe64f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.359995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4de0ef-9b72-4311-9ec2-edb2d757588a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.399446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a837a-3efe-44c5-a9e1-c610574db576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.400335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a837a-3efe-44c5-a9e1-c610574db576 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.400667 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a837a-3efe-44c5-a9e1-c610574db576 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.400995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a837a-3efe-44c5-a9e1-c610574db576 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.401292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d5a01-0f2b-4407-b521-e33cc3da47f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.402105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d5a01-0f2b-4407-b521-e33cc3da47f3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.402200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1a837a-3efe-44c5-a9e1-c610574db576 None admin] RBAC: Adding query filter params (role.id=a638e91b70624f669ffe76dac32323f5, scope.project.id=8cea8011767244e0a581010033a1caf4, user.id=f8c83e570c724b1f97ec66b5d5233ecb) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:27.402422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d5a01-0f2b-4407-b521-e33cc3da47f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.402566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1a837a-3efe-44c5-a9e1-c610574db576 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.402701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d5a01-0f2b-4407-b521-e33cc3da47f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.404205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1a837a-3efe-44c5-a9e1-c610574db576 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.411834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 140/277] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/role_assignments?user.id=f8c83e570c724b1f97ec66b5d5233ecb&scope.project.id=8cea8011767244e0a581010033a1caf4&role.id=a638e91b70624f669ffe76dac32323f5 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:27.412843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a837a-3efe-44c5-a9e1-c610574db576 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.427751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d5a01-0f2b-4407-b521-e33cc3da47f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-0e5d5a01-0f2b-4407-b521-e33cc3da47f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:27.437302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:27.438280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:27.439111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:27.439111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:27.439111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:27.439111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:27.439111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:27.439111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:27.439111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:27.439111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:27.439111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:27.439111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:27.439111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:27.439111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:27.439111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 138/278] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:27.440296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d5a01-0f2b-4407-b521-e33cc3da47f3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.576685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d5a01-0f2b-4407-b521-e33cc3da47f3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.587856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a837a-3efe-44c5-a9e1-c610574db576 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.594457 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.623852 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb5be9-aeec-4df0-a231-c5a50b4c6b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.624548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb5be9-aeec-4df0-a231-c5a50b4c6b2d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.624838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb5be9-aeec-4df0-a231-c5a50b4c6b2d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.625121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb5be9-aeec-4df0-a231-c5a50b4c6b2d None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.625878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb5be9-aeec-4df0-a231-c5a50b4c6b2d None admin] RBAC: Adding query filter params (name=member) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:27.626192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb5be9-aeec-4df0-a231-c5a50b4c6b2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.627793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb5be9-aeec-4df0-a231-c5a50b4c6b2d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.633544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.638380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 139/279] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:27.640732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6945089-bd47-4bdc-94b0-830ef37e2535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.641500 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6945089-bd47-4bdc-94b0-830ef37e2535 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.641839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6945089-bd47-4bdc-94b0-830ef37e2535 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.642155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6945089-bd47-4bdc-94b0-830ef37e2535 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.643118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6945089-bd47-4bdc-94b0-830ef37e2535 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.645032 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb5be9-aeec-4df0-a231-c5a50b4c6b2d None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.646783 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6945089-bd47-4bdc-94b0-830ef37e2535 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.772660 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 141/280] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:27 2022] POST /identity/v3/users => generated 293 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:27.773306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6945089-bd47-4bdc-94b0-830ef37e2535 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.776119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb5be9-aeec-4df0-a231-c5a50b4c6b2d None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.829180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bc303-f140-44b3-a39a-2379e13d287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.830772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bc303-f140-44b3-a39a-2379e13d287b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.831442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bc303-f140-44b3-a39a-2379e13d287b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.832089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bc303-f140-44b3-a39a-2379e13d287b None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.834059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bc303-f140-44b3-a39a-2379e13d287b None admin] RBAC: Adding query filter params (name=reader) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:27.834687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bc303-f140-44b3-a39a-2379e13d287b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.837438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bc303-f140-44b3-a39a-2379e13d287b None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.850425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 140/281] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:27.857736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bc303-f140-44b3-a39a-2379e13d287b None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:27.902135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:27.910586 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6945089-bd47-4bdc-94b0-830ef37e2535 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.951007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e15b0-833a-4b74-8b5c-6414adc8bd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:27.952386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e15b0-833a-4b74-8b5c-6414adc8bd54 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:27.952880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e15b0-833a-4b74-8b5c-6414adc8bd54 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:27.953357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e15b0-833a-4b74-8b5c-6414adc8bd54 None admin] PATH_INFO: `/v3/users/668fea2611474386a5d869d7778c84d9` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:27.970874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e15b0-833a-4b74-8b5c-6414adc8bd54 None admin] RBAC: Authorizing `identity:get_user(user_id=668fea2611474386a5d869d7778c84d9)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:27.972990 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e15b0-833a-4b74-8b5c-6414adc8bd54 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:27.976209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bc303-f140-44b3-a39a-2379e13d287b None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:27.976816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 142/282] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/users/668fea2611474386a5d869d7778c84d9 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:27.983667 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e15b0-833a-4b74-8b5c-6414adc8bd54 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.033148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45630d-8018-45b2-8b76-97019bc1b553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.034178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45630d-8018-45b2-8b76-97019bc1b553 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.034501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45630d-8018-45b2-8b76-97019bc1b553 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.034779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45630d-8018-45b2-8b76-97019bc1b553 None admin] PATH_INFO: `/v3/users/09adf329fc2a484092b29e52cf4c6c0f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.036482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45630d-8018-45b2-8b76-97019bc1b553 None admin] RBAC: Authorizing `identity:get_user(user_id=09adf329fc2a484092b29e52cf4c6c0f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.037981 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45630d-8018-45b2-8b76-97019bc1b553 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:28.040631 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 141/283] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/users/09adf329fc2a484092b29e52cf4c6c0f => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:28.047962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45630d-8018-45b2-8b76-97019bc1b553 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.054076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.127153 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e15b0-833a-4b74-8b5c-6414adc8bd54 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.178658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b025ab3-64cd-4b3a-a03c-6141d0897bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.179352 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b025ab3-64cd-4b3a-a03c-6141d0897bc9 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.179642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b025ab3-64cd-4b3a-a03c-6141d0897bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.179926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b025ab3-64cd-4b3a-a03c-6141d0897bc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.180812 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b025ab3-64cd-4b3a-a03c-6141d0897bc9 None admin] RBAC: Adding query filter params (role.id=9dc988eb162e47e48df810053eecb165, user.id=668fea2611474386a5d869d7778c84d9, scope.system=all) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:28.181133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b025ab3-64cd-4b3a-a03c-6141d0897bc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.182615 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b025ab3-64cd-4b3a-a03c-6141d0897bc9 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:28.190405 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 143/284] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Jul 25 18:42:27 2022] GET /identity/v3/role_assignments?user.id=668fea2611474386a5d869d7778c84d9&scope.system=all&role.id=9dc988eb162e47e48df810053eecb165 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:28.190405 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d8274-7715-4a0b-873f-d078c6b72475 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.190813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d8274-7715-4a0b-873f-d078c6b72475 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.191947 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d8274-7715-4a0b-873f-d078c6b72475 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.193006 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 144/285] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:28.199360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974786d-7692-4a75-be5a-24b5e0d0a539 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.199753 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974786d-7692-4a75-be5a-24b5e0d0a539 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.199951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974786d-7692-4a75-be5a-24b5e0d0a539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.202211 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45630d-8018-45b2-8b76-97019bc1b553 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.205966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.221432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-4974786d-7692-4a75-be5a-24b5e0d0a539 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:28.222964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4974786d-7692-4a75-be5a-24b5e0d0a539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:28.265272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae2f81-1ba3-4d33-933b-6253537682e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.266168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae2f81-1ba3-4d33-933b-6253537682e3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.266488 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae2f81-1ba3-4d33-933b-6253537682e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.266774 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae2f81-1ba3-4d33-933b-6253537682e3 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.268807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae2f81-1ba3-4d33-933b-6253537682e3 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.270393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae2f81-1ba3-4d33-933b-6253537682e3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:28.272596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 142/286] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:28 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:28.283429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae2f81-1ba3-4d33-933b-6253537682e3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.325632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 145/287] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:28.417751 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974786d-7692-4a75-be5a-24b5e0d0a539 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.470598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae2f81-1ba3-4d33-933b-6253537682e3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.495933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.552526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b9cd1-232b-4479-9539-99cce3758327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.554263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b9cd1-232b-4479-9539-99cce3758327 None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.554832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b9cd1-232b-4479-9539-99cce3758327 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.555406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b9cd1-232b-4479-9539-99cce3758327 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4/users/09adf329fc2a484092b29e52cf4c6c0f/roles/b2f299f9533944ca821a102e13bd4c69` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.559856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b9cd1-232b-4479-9539-99cce3758327 None admin] RBAC: Authorizing `identity:create_grant(project_id=8cea8011767244e0a581010033a1caf4, user_id=09adf329fc2a484092b29e52cf4c6c0f, role_id=b2f299f9533944ca821a102e13bd4c69)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.561623 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b9cd1-232b-4479-9539-99cce3758327 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:28.584928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974786d-7692-4a75-be5a-24b5e0d0a539 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.627083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193b41e-602a-4a27-9395-fb5ce857cb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.627978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193b41e-602a-4a27-9395-fb5ce857cb27 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.628270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193b41e-602a-4a27-9395-fb5ce857cb27 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.628548 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193b41e-602a-4a27-9395-fb5ce857cb27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.640785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193b41e-602a-4a27-9395-fb5ce857cb27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-e193b41e-602a-4a27-9395-fb5ce857cb27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:28.649390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:28.650514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:28.651656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:28.651656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:28.651656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:28.651656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:28.651656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:28.651656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:28.651656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:28.651656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:28.651656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:28.651656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:28.651656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:28.651656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:28.651656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 146/288] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:28.661832 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193b41e-602a-4a27-9395-fb5ce857cb27 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.667075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 143/289] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:42:28 2022] PUT /identity/v3/projects/8cea8011767244e0a581010033a1caf4/users/09adf329fc2a484092b29e52cf4c6c0f/roles/b2f299f9533944ca821a102e13bd4c69 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:28.779804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.783926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193b41e-602a-4a27-9395-fb5ce857cb27 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:28.824518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5914f-f441-43b7-b103-535326e25052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:28.825227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5914f-f441-43b7-b103-535326e25052 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:28.825571 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5914f-f441-43b7-b103-535326e25052 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:28.825866 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5914f-f441-43b7-b103-535326e25052 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:28.826697 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5914f-f441-43b7-b103-535326e25052 None admin] RBAC: Adding query filter params (name=member) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:28.827017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5914f-f441-43b7-b103-535326e25052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:28.828470 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5914f-f441-43b7-b103-535326e25052 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:28.839433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 147/290] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:28.847820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b9cd1-232b-4479-9539-99cce3758327 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:28.900616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:28.989639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b9cd1-232b-4479-9539-99cce3758327 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:29.029246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ee415-3a57-47bf-99ed-08c36c07bbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:29.030346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ee415-3a57-47bf-99ed-08c36c07bbf5 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:29.030592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ee415-3a57-47bf-99ed-08c36c07bbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:29.030864 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ee415-3a57-47bf-99ed-08c36c07bbf5 None admin] PATH_INFO: `/v3/users/9b2c12ba651743729eda4429bad7652b` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:29.032328 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ee415-3a57-47bf-99ed-08c36c07bbf5 None admin] RBAC: Authorizing `identity:get_user(user_id=9b2c12ba651743729eda4429bad7652b)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:29.033824 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ee415-3a57-47bf-99ed-08c36c07bbf5 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:29.035541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 144/291] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:28 2022] GET /identity/v3/users/9b2c12ba651743729eda4429bad7652b => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:29.043657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5914f-f441-43b7-b103-535326e25052 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:29.218075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5914f-f441-43b7-b103-535326e25052 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:29.277115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69f8dd-91ea-424e-bbf1-6694d27c4e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:29.277823 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69f8dd-91ea-424e-bbf1-6694d27c4e0d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:29.278111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69f8dd-91ea-424e-bbf1-6694d27c4e0d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:29.278456 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69f8dd-91ea-424e-bbf1-6694d27c4e0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:29.325805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:29.384133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69f8dd-91ea-424e-bbf1-6694d27c4e0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:29.385597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69f8dd-91ea-424e-bbf1-6694d27c4e0d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:29.542442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 148/292] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:29.543956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ee415-3a57-47bf-99ed-08c36c07bbf5 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:29.656681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ee415-3a57-47bf-99ed-08c36c07bbf5 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:29.666741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:29.705501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30229b-57e8-4f58-8bdc-eedfa0c064e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:29.706423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30229b-57e8-4f58-8bdc-eedfa0c064e2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:29.706744 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30229b-57e8-4f58-8bdc-eedfa0c064e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:29.707021 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30229b-57e8-4f58-8bdc-eedfa0c064e2 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:29.708830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30229b-57e8-4f58-8bdc-eedfa0c064e2 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:29.710378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30229b-57e8-4f58-8bdc-eedfa0c064e2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:29.712187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 145/293] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:29 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:29.718550 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69f8dd-91ea-424e-bbf1-6694d27c4e0d None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:29.823970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:29.888055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69f8dd-91ea-424e-bbf1-6694d27c4e0d None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:29.946384 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a599222-46e6-4ff6-91f6-298e8bf5d3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:29.947443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a599222-46e6-4ff6-91f6-298e8bf5d3e5 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:29.947881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a599222-46e6-4ff6-91f6-298e8bf5d3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:29.948402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a599222-46e6-4ff6-91f6-298e8bf5d3e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:29.949562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a599222-46e6-4ff6-91f6-298e8bf5d3e5 None admin] RBAC: Adding query filter params (role.id=9dc988eb162e47e48df810053eecb165, scope.project.id=8cea8011767244e0a581010033a1caf4, user.id=9b2c12ba651743729eda4429bad7652b) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:29.950108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a599222-46e6-4ff6-91f6-298e8bf5d3e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:29.952003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a599222-46e6-4ff6-91f6-298e8bf5d3e5 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:29.960589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 149/294] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:29 2022] GET /identity/v3/role_assignments?user.id=9b2c12ba651743729eda4429bad7652b&scope.project.id=8cea8011767244e0a581010033a1caf4&role.id=9dc988eb162e47e48df810053eecb165 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:30.622272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccbcfa-0276-42b9-8f34-7fe4308a51cb None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:30.622810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccbcfa-0276-42b9-8f34-7fe4308a51cb None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:30.622919 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccbcfa-0276-42b9-8f34-7fe4308a51cb None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:30.624606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 146/295] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:30.632257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad5a5-52ec-403f-b53b-4d318d4eb652 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:30.632766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad5a5-52ec-403f-b53b-4d318d4eb652 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:30.632961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad5a5-52ec-403f-b53b-4d318d4eb652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:30.654191 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-4a2ad5a5-52ec-403f-b53b-4d318d4eb652 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:30.655484 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4a2ad5a5-52ec-403f-b53b-4d318d4eb652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:30.668985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e790d-cdb4-46cf-bfbb-71071cf4f21f None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:30.669308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e790d-cdb4-46cf-bfbb-71071cf4f21f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:30.669561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e790d-cdb4-46cf-bfbb-71071cf4f21f None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:30.671204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 147/296] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:30.680703 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62075b-dc67-475a-9b9e-7a70d5490e30 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:30.681008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62075b-dc67-475a-9b9e-7a70d5490e30 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:30.681282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62075b-dc67-475a-9b9e-7a70d5490e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:30.711543 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-6a62075b-dc67-475a-9b9e-7a70d5490e30 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:30.713237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6a62075b-dc67-475a-9b9e-7a70d5490e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:30.760518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 150/297] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:30.786602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ad5a5-52ec-403f-b53b-4d318d4eb652 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:30.792585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 148/298] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:30.816829 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62075b-dc67-475a-9b9e-7a70d5490e30 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:30.867648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:30.924838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ad5a5-52ec-403f-b53b-4d318d4eb652 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:30.936030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:30.940108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62075b-dc67-475a-9b9e-7a70d5490e30 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:30.963224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41b20d-dd16-4d20-bab5-8a87f77cd559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:30.964028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41b20d-dd16-4d20-bab5-8a87f77cd559 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:30.964317 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41b20d-dd16-4d20-bab5-8a87f77cd559 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:30.964709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41b20d-dd16-4d20-bab5-8a87f77cd559 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:30.965824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41b20d-dd16-4d20-bab5-8a87f77cd559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:30.967314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41b20d-dd16-4d20-bab5-8a87f77cd559 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:30.974449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 151/299] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:30.974449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7596b-240f-4b9c-bdfd-c80c8624fb5e None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:30.974449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7596b-240f-4b9c-bdfd-c80c8624fb5e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:30.974449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7596b-240f-4b9c-bdfd-c80c8624fb5e None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:30.974449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 152/300] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:30.979146 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7596b-240f-4b9c-bdfd-c80c8624fb5e None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:30.997362 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978903ab-b4ab-42f9-a15a-5a68cc3c7cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:30.998406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978903ab-b4ab-42f9-a15a-5a68cc3c7cd9 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:30.998729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978903ab-b4ab-42f9-a15a-5a68cc3c7cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:30.999019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978903ab-b4ab-42f9-a15a-5a68cc3c7cd9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.012062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978903ab-b4ab-42f9-a15a-5a68cc3c7cd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-978903ab-b4ab-42f9-a15a-5a68cc3c7cd9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:31.021922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:31.023023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:31.023833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:31.023833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:31.023833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:31.023833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:31.023833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:31.023833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:31.023833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:31.023833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:31.023833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:31.023833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:31.023833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:31.023833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:31.023833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 149/301] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:31.025049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f925638-4ccf-4e68-8e93-fe51f83f1369 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.025049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f925638-4ccf-4e68-8e93-fe51f83f1369 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.025381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f925638-4ccf-4e68-8e93-fe51f83f1369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.046070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-9f925638-4ccf-4e68-8e93-fe51f83f1369 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:31.047461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9f925638-4ccf-4e68-8e93-fe51f83f1369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:31.105559 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7596b-240f-4b9c-bdfd-c80c8624fb5e None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:31.133333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:31.136279 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 150/302] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:31.137171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f925638-4ccf-4e68-8e93-fe51f83f1369 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:31.164328 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de749e-f970-476e-b4f3-7e4b0e03febc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:31.165254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de749e-f970-476e-b4f3-7e4b0e03febc None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.165393 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de749e-f970-476e-b4f3-7e4b0e03febc None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.165653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de749e-f970-476e-b4f3-7e4b0e03febc None admin] PATH_INFO: `/v3/groups` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.166383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de749e-f970-476e-b4f3-7e4b0e03febc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:31.168035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de749e-f970-476e-b4f3-7e4b0e03febc None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:31.235028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 153/303] 10.222.0.17 () {60 vars in 1134 bytes} [Mon Jul 25 18:42:30 2022] POST /identity/v3/groups => generated 229 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:31.236301 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de749e-f970-476e-b4f3-7e4b0e03febc None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:31.237022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:31.264927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f925638-4ccf-4e68-8e93-fe51f83f1369 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:31.321765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9b128-b019-4517-b73a-253318a581eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:31.323139 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9b128-b019-4517-b73a-253318a581eb None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.323710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9b128-b019-4517-b73a-253318a581eb None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.324367 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9b128-b019-4517-b73a-253318a581eb None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.326306 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9b128-b019-4517-b73a-253318a581eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:31.326844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9b128-b019-4517-b73a-253318a581eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:31.328703 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9b128-b019-4517-b73a-253318a581eb None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:31.355437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 151/304] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:31.355437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fc9e0-b360-438a-bf3c-37a4efea93c6 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.355437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fc9e0-b360-438a-bf3c-37a4efea93c6 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.355437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fc9e0-b360-438a-bf3c-37a4efea93c6 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.356784 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 152/305] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:31.359166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8fc9e0-b360-438a-bf3c-37a4efea93c6 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:31.391388 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de749e-f970-476e-b4f3-7e4b0e03febc None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:31.437951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb55e01-2006-4450-898a-6209843ea984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:31.438818 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb55e01-2006-4450-898a-6209843ea984 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.439104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb55e01-2006-4450-898a-6209843ea984 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.439393 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb55e01-2006-4450-898a-6209843ea984 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.453737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb55e01-2006-4450-898a-6209843ea984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-fdb55e01-2006-4450-898a-6209843ea984 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:31.462889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:31.463863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:31.465338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:31.465338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:31.465338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:31.465338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:31.465338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:31.465338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:31.465338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:31.465338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:31.465338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:31.465338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:31.465338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:31.465338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:31.465338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 154/306] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:31.467522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5574d5-b6e4-4ff2-810a-7c0137072df3 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.467522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5574d5-b6e4-4ff2-810a-7c0137072df3 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.467522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5574d5-b6e4-4ff2-810a-7c0137072df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.485829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-8d5574d5-b6e4-4ff2-810a-7c0137072df3 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:31.486918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8d5574d5-b6e4-4ff2-810a-7c0137072df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:31.499111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8fc9e0-b360-438a-bf3c-37a4efea93c6 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:31.532089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:31.542494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8ecaa-b83c-4a4d-a052-66359f0cd037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:31.543651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8ecaa-b83c-4a4d-a052-66359f0cd037 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.543972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8ecaa-b83c-4a4d-a052-66359f0cd037 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.544251 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8ecaa-b83c-4a4d-a052-66359f0cd037 None admin] PATH_INFO: `/v3/users/0726b1b313114751a97d0f7ab41f8803` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.561735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8ecaa-b83c-4a4d-a052-66359f0cd037 None admin] RBAC: Authorizing `identity:get_user(user_id=0726b1b313114751a97d0f7ab41f8803)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:31.563486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8ecaa-b83c-4a4d-a052-66359f0cd037 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:31.565844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 153/307] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:31 2022] GET /identity/v3/users/0726b1b313114751a97d0f7ab41f8803 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:31.566757 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8ecaa-b83c-4a4d-a052-66359f0cd037 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:31.580356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:31.583678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 155/308] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:31.584482 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5574d5-b6e4-4ff2-810a-7c0137072df3 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:31.695647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8ecaa-b83c-4a4d-a052-66359f0cd037 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:31.718407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5574d5-b6e4-4ff2-810a-7c0137072df3 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:31.736411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aebdfa-e95e-43d8-9db1-2deb87331ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:31.737023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aebdfa-e95e-43d8-9db1-2deb87331ea1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.737313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aebdfa-e95e-43d8-9db1-2deb87331ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.737595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aebdfa-e95e-43d8-9db1-2deb87331ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.738393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aebdfa-e95e-43d8-9db1-2deb87331ea1 None admin] RBAC: Adding query filter params (name=member) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:31.738831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aebdfa-e95e-43d8-9db1-2deb87331ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:31.741071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aebdfa-e95e-43d8-9db1-2deb87331ea1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:31.750142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 154/309] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:31.750142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aebdfa-e95e-43d8-9db1-2deb87331ea1 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:31.772640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914d10c-44d8-40c5-82ca-2dc955743ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:31.773643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914d10c-44d8-40c5-82ca-2dc955743ef3 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.774104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914d10c-44d8-40c5-82ca-2dc955743ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.774438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914d10c-44d8-40c5-82ca-2dc955743ef3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.775483 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914d10c-44d8-40c5-82ca-2dc955743ef3 None admin] RBAC: Adding query filter params (role.id=b2f299f9533944ca821a102e13bd4c69, user.id=0726b1b313114751a97d0f7ab41f8803, scope.system=all) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:31.775889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914d10c-44d8-40c5-82ca-2dc955743ef3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:31.777780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914d10c-44d8-40c5-82ca-2dc955743ef3 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:31.784923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 156/310] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Jul 25 18:42:31 2022] GET /identity/v3/role_assignments?user.id=0726b1b313114751a97d0f7ab41f8803&scope.system=all&role.id=b2f299f9533944ca821a102e13bd4c69 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:31.785775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914d10c-44d8-40c5-82ca-2dc955743ef3 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:31.838910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:31.847480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:31.888946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aebdfa-e95e-43d8-9db1-2deb87331ea1 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:31.933053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ebd3a-ed5a-4e2c-adbd-78f5c5d69578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:31.934435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ebd3a-ed5a-4e2c-adbd-78f5c5d69578 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.934970 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ebd3a-ed5a-4e2c-adbd-78f5c5d69578 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.935515 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ebd3a-ed5a-4e2c-adbd-78f5c5d69578 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.936295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914d10c-44d8-40c5-82ca-2dc955743ef3 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:31.949656 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ebd3a-ed5a-4e2c-adbd-78f5c5d69578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-451ebd3a-ed5a-4e2c-adbd-78f5c5d69578 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:31.960173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:31.961372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:31.962244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:31.962244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:31.962244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:31.962244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:31.962244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:31.962244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:31.962244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:31.962244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:31.962244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:31.962244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:31.962244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:31.962244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:31.962244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 155/311] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:31.969332 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ebd3a-ed5a-4e2c-adbd-78f5c5d69578 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:31.983956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523b546-d62d-409c-8eac-f5577109464e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:31.984852 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523b546-d62d-409c-8eac-f5577109464e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:31.985146 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523b546-d62d-409c-8eac-f5577109464e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:31.985424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523b546-d62d-409c-8eac-f5577109464e None admin] PATH_INFO: `/v3/users/668fea2611474386a5d869d7778c84d9` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:31.986763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523b546-d62d-409c-8eac-f5577109464e None admin] RBAC: Authorizing `identity:get_user(user_id=668fea2611474386a5d869d7778c84d9)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:31.988150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523b546-d62d-409c-8eac-f5577109464e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:31.990310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 157/312] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:31 2022] GET /identity/v3/users/668fea2611474386a5d869d7778c84d9 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:31.999483 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523b546-d62d-409c-8eac-f5577109464e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:32.088235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ebd3a-ed5a-4e2c-adbd-78f5c5d69578 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:32.122109 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:32.128259 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a066e-3b5f-49c7-9c48-cf8b639d9aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:32.136619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a066e-3b5f-49c7-9c48-cf8b639d9aca None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.136619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a066e-3b5f-49c7-9c48-cf8b639d9aca None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.136619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a066e-3b5f-49c7-9c48-cf8b639d9aca None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.136619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a066e-3b5f-49c7-9c48-cf8b639d9aca None admin] RBAC: Adding query filter params (name=reader) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:32.136619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a066e-3b5f-49c7-9c48-cf8b639d9aca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:32.136619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a066e-3b5f-49c7-9c48-cf8b639d9aca None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:32.140762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523b546-d62d-409c-8eac-f5577109464e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:32.143474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 156/313] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:32.143932 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:32.150707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a066e-3b5f-49c7-9c48-cf8b639d9aca None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:32.195770 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a97b8-8750-4a58-94f1-056e7dfdb22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:32.197032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a97b8-8750-4a58-94f1-056e7dfdb22e None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.197547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a97b8-8750-4a58-94f1-056e7dfdb22e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.197992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a97b8-8750-4a58-94f1-056e7dfdb22e None admin] PATH_INFO: `/v3/system/users/668fea2611474386a5d869d7778c84d9/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.199966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a97b8-8750-4a58-94f1-056e7dfdb22e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=668fea2611474386a5d869d7778c84d9, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:32.201907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a97b8-8750-4a58-94f1-056e7dfdb22e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:32.271387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a066e-3b5f-49c7-9c48-cf8b639d9aca None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:32.297176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 158/314] 10.222.0.17 () {58 vars in 1334 bytes} [Mon Jul 25 18:42:31 2022] PUT /identity/v3/system/users/668fea2611474386a5d869d7778c84d9/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:32.324540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0fd30-db29-47be-a849-5bee43eb714b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:32.326292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0fd30-db29-47be-a849-5bee43eb714b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.326699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0fd30-db29-47be-a849-5bee43eb714b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.327197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0fd30-db29-47be-a849-5bee43eb714b None admin] PATH_INFO: `/v3/users/09adf329fc2a484092b29e52cf4c6c0f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.328984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0fd30-db29-47be-a849-5bee43eb714b None admin] RBAC: Authorizing `identity:get_user(user_id=09adf329fc2a484092b29e52cf4c6c0f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:32.330988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0fd30-db29-47be-a849-5bee43eb714b None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:32.333659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 157/315] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:32 2022] GET /identity/v3/users/09adf329fc2a484092b29e52cf4c6c0f => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:32.339909 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a97b8-8750-4a58-94f1-056e7dfdb22e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:32.478376 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a97b8-8750-4a58-94f1-056e7dfdb22e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:32.529290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1a20e-404c-47df-ae51-825d1915f43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:32.530064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1a20e-404c-47df-ae51-825d1915f43d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:32.530392 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1a20e-404c-47df-ae51-825d1915f43d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:32.530698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1a20e-404c-47df-ae51-825d1915f43d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:32.586551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:32.633079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1a20e-404c-47df-ae51-825d1915f43d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:32.634605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1a20e-404c-47df-ae51-825d1915f43d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:32.809824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 159/316] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:32.815379 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0fd30-db29-47be-a849-5bee43eb714b None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:32.902093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:32.964491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0fd30-db29-47be-a849-5bee43eb714b None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:33.009413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7930fd6-4bb5-497e-a061-94c99d30d162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:33.010312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7930fd6-4bb5-497e-a061-94c99d30d162 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.010634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7930fd6-4bb5-497e-a061-94c99d30d162 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.010911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7930fd6-4bb5-497e-a061-94c99d30d162 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.012230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7930fd6-4bb5-497e-a061-94c99d30d162 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:33.013775 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7930fd6-4bb5-497e-a061-94c99d30d162 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:33.015992 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 158/317] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:32 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:33.022028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1a20e-404c-47df-ae51-825d1915f43d None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:33.097330 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:33.139965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1a20e-404c-47df-ae51-825d1915f43d None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:33.181557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475f203-cc55-4d1b-a90e-b5e4b7401ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:33.182242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475f203-cc55-4d1b-a90e-b5e4b7401ce7 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.182625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475f203-cc55-4d1b-a90e-b5e4b7401ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.182915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475f203-cc55-4d1b-a90e-b5e4b7401ce7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.183819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475f203-cc55-4d1b-a90e-b5e4b7401ce7 None admin] RBAC: Adding query filter params (role.id=b2f299f9533944ca821a102e13bd4c69, scope.project.id=8cea8011767244e0a581010033a1caf4, user.id=09adf329fc2a484092b29e52cf4c6c0f) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:33.184146 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475f203-cc55-4d1b-a90e-b5e4b7401ce7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:33.185620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475f203-cc55-4d1b-a90e-b5e4b7401ce7 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:33.192816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 160/318] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:42:33 2022] GET /identity/v3/role_assignments?user.id=09adf329fc2a484092b29e52cf4c6c0f&scope.project.id=8cea8011767244e0a581010033a1caf4&role.id=b2f299f9533944ca821a102e13bd4c69 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:33.434935 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f034a5-57b9-41ab-ba4e-b66db737598b None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.435671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f034a5-57b9-41ab-ba4e-b66db737598b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.436221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f034a5-57b9-41ab-ba4e-b66db737598b None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.437878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 159/319] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:33.444798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9b808-1dd9-4b0d-a021-81980e0f6dfb None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.445428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9b808-1dd9-4b0d-a021-81980e0f6dfb None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.445902 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9b808-1dd9-4b0d-a021-81980e0f6dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.471717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-93a9b808-1dd9-4b0d-a021-81980e0f6dfb None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:33.474169 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-93a9b808-1dd9-4b0d-a021-81980e0f6dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:33.556983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 161/320] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:33.580340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f034a5-57b9-41ab-ba4e-b66db737598b None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:33.711824 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f034a5-57b9-41ab-ba4e-b66db737598b None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:33.748602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:33.758747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47049f09-191b-42cf-a13c-af063a2b9f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:33.763554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47049f09-191b-42cf-a13c-af063a2b9f24 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.763554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47049f09-191b-42cf-a13c-af063a2b9f24 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.763554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47049f09-191b-42cf-a13c-af063a2b9f24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.763554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47049f09-191b-42cf-a13c-af063a2b9f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:33.763848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47049f09-191b-42cf-a13c-af063a2b9f24 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:33.766005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 160/321] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:33.774405 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9b808-1dd9-4b0d-a021-81980e0f6dfb None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:33.856295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:33.904313 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9b808-1dd9-4b0d-a021-81980e0f6dfb None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:33.944516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7422c-6eba-4f16-ad92-631b7f101a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:33.945266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7422c-6eba-4f16-ad92-631b7f101a50 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.945538 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7422c-6eba-4f16-ad92-631b7f101a50 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.945824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7422c-6eba-4f16-ad92-631b7f101a50 None admin] PATH_INFO: `/v3/groups` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.946569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7422c-6eba-4f16-ad92-631b7f101a50 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:33.948059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7422c-6eba-4f16-ad92-631b7f101a50 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:33.972943 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4954906-3450-4e0f-9e63-c85bd979746a None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.973505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4954906-3450-4e0f-9e63-c85bd979746a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.973505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4954906-3450-4e0f-9e63-c85bd979746a None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:33.974994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 161/322] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:33.980880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c5685-da84-42ff-b0f9-e61b54ff83a9 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:33.981174 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c5685-da84-42ff-b0f9-e61b54ff83a9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:33.981469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c5685-da84-42ff-b0f9-e61b54ff83a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:34.002434 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-e47c5685-da84-42ff-b0f9-e61b54ff83a9 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:34.003819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e47c5685-da84-42ff-b0f9-e61b54ff83a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:34.094624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 162/323] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:34.096396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 162/324] 10.222.0.17 () {60 vars in 1134 bytes} [Mon Jul 25 18:42:33 2022] POST /identity/v3/groups => generated 226 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:34.126838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c5685-da84-42ff-b0f9-e61b54ff83a9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:34.307718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c5685-da84-42ff-b0f9-e61b54ff83a9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:34.366167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:34.379597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8bee8-bbd1-446c-8e36-d6c98ba0a49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:34.381466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8bee8-bbd1-446c-8e36-d6c98ba0a49c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:34.381917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8bee8-bbd1-446c-8e36-d6c98ba0a49c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:34.382221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8bee8-bbd1-446c-8e36-d6c98ba0a49c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:34.397408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8bee8-bbd1-446c-8e36-d6c98ba0a49c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-fbb8bee8-bbd1-446c-8e36-d6c98ba0a49c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:34.406525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:34.408426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:34.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:34.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:34.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:34.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:34.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:34.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:34.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:34.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:34.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:34.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:34.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:34.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:34.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 163/325] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:34.421938 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7422c-6eba-4f16-ad92-631b7f101a50 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:34.574823 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7422c-6eba-4f16-ad92-631b7f101a50 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:34.607670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:34.624866 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5bbd2-2e06-4a11-8dc8-a8503d30cdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:34.625986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5bbd2-2e06-4a11-8dc8-a8503d30cdee None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:34.626633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5bbd2-2e06-4a11-8dc8-a8503d30cdee None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:34.627276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5bbd2-2e06-4a11-8dc8-a8503d30cdee None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:34.636263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5bbd2-2e06-4a11-8dc8-a8503d30cdee None admin] RBAC: Adding query filter params (name=reader) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:34.636263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5bbd2-2e06-4a11-8dc8-a8503d30cdee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:34.636263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5bbd2-2e06-4a11-8dc8-a8503d30cdee None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:34.643339 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 163/326] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:34.651125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8bee8-bbd1-446c-8e36-d6c98ba0a49c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:34.795941 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf85a-1392-4ec3-b84c-a47e69188c09 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:34.796423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf85a-1392-4ec3-b84c-a47e69188c09 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:34.796607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf85a-1392-4ec3-b84c-a47e69188c09 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:34.797926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 164/327] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:34.804970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aa375-5013-4b2e-a88d-1a3952cae9a1 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:34.805368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aa375-5013-4b2e-a88d-1a3952cae9a1 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:34.805548 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aa375-5013-4b2e-a88d-1a3952cae9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:34.823969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8bee8-bbd1-446c-8e36-d6c98ba0a49c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:34.825554 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c37aa375-5013-4b2e-a88d-1a3952cae9a1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:34.826782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c37aa375-5013-4b2e-a88d-1a3952cae9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:34.900171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c8e31-670d-4745-872c-b01f938b68c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:34.901126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8e31-670d-4745-872c-b01f938b68c6 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:34.901420 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8e31-670d-4745-872c-b01f938b68c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:34.901710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8e31-670d-4745-872c-b01f938b68c6 None admin] PATH_INFO: `/v3/users/0726b1b313114751a97d0f7ab41f8803` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:34.903117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c8e31-670d-4745-872c-b01f938b68c6 None admin] RBAC: Authorizing `identity:get_user(user_id=0726b1b313114751a97d0f7ab41f8803)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:34.904880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c8e31-670d-4745-872c-b01f938b68c6 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:34.907511 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 164/328] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:34 2022] GET /identity/v3/users/0726b1b313114751a97d0f7ab41f8803 => generated 293 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:34.918529 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c8e31-670d-4745-872c-b01f938b68c6 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:34.945628 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 165/329] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:34.973769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37aa375-5013-4b2e-a88d-1a3952cae9a1 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:34.993523 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:35.046639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c8e31-670d-4745-872c-b01f938b68c6 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:35.104413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed5abe-e07b-46b6-a9a5-4b751185719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:35.105351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed5abe-e07b-46b6-a9a5-4b751185719e None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:35.105645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed5abe-e07b-46b6-a9a5-4b751185719e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:35.105928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed5abe-e07b-46b6-a9a5-4b751185719e None admin] PATH_INFO: `/v3/system/users/0726b1b313114751a97d0f7ab41f8803/roles/b2f299f9533944ca821a102e13bd4c69` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:35.108372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed5abe-e07b-46b6-a9a5-4b751185719e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0726b1b313114751a97d0f7ab41f8803, role_id=b2f299f9533944ca821a102e13bd4c69)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:35.109817 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed5abe-e07b-46b6-a9a5-4b751185719e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:35.110824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37aa375-5013-4b2e-a88d-1a3952cae9a1 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:35.142366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:35.162362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec5dbf-ca72-4680-9900-ef7b028208cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:35.162362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec5dbf-ca72-4680-9900-ef7b028208cc None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:35.162362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec5dbf-ca72-4680-9900-ef7b028208cc None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:35.162362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec5dbf-ca72-4680-9900-ef7b028208cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:35.174037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec5dbf-ca72-4680-9900-ef7b028208cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-06ec5dbf-ca72-4680-9900-ef7b028208cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:35.183230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:35.184241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:35.185208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:35.185208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:35.185208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:35.185208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:35.185208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:35.185208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:35.185208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:35.185208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:35.185208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:35.185208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:35.185208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:35.185208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:35.187266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 166/330] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:35.189355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 165/331] 10.222.0.17 () {58 vars in 1334 bytes} [Mon Jul 25 18:42:34 2022] PUT /identity/v3/system/users/0726b1b313114751a97d0f7ab41f8803/roles/b2f299f9533944ca821a102e13bd4c69 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:35.196286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec5dbf-ca72-4680-9900-ef7b028208cc None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:35.344230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec5dbf-ca72-4680-9900-ef7b028208cc None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:35.397985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40c2ec-9a89-4992-a2bf-0b391158a4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:35.399075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40c2ec-9a89-4992-a2bf-0b391158a4bd None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:35.399567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40c2ec-9a89-4992-a2bf-0b391158a4bd None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:35.400047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40c2ec-9a89-4992-a2bf-0b391158a4bd None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:35.401114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40c2ec-9a89-4992-a2bf-0b391158a4bd None admin] RBAC: Adding query filter params (name=member) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:35.401649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40c2ec-9a89-4992-a2bf-0b391158a4bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:35.403370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40c2ec-9a89-4992-a2bf-0b391158a4bd None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:35.417658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 167/332] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:35.424253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed5abe-e07b-46b6-a9a5-4b751185719e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:35.569458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed5abe-e07b-46b6-a9a5-4b751185719e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:35.594564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:35.621464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc4074-d8a0-4a94-942b-c327c0143e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:35.622509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc4074-d8a0-4a94-942b-c327c0143e09 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:35.622827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc4074-d8a0-4a94-942b-c327c0143e09 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:35.623109 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc4074-d8a0-4a94-942b-c327c0143e09 None admin] PATH_INFO: `/v3/users/668fea2611474386a5d869d7778c84d9` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:35.624481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc4074-d8a0-4a94-942b-c327c0143e09 None admin] RBAC: Authorizing `identity:get_user(user_id=668fea2611474386a5d869d7778c84d9)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:35.626192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc4074-d8a0-4a94-942b-c327c0143e09 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:35.628611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 166/333] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:35 2022] GET /identity/v3/users/668fea2611474386a5d869d7778c84d9 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:35.635758 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40c2ec-9a89-4992-a2bf-0b391158a4bd None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:35.711561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:35.797628 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40c2ec-9a89-4992-a2bf-0b391158a4bd None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:35.858079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bb341-afc3-423a-9fca-30e6177a469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:35.858792 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bb341-afc3-423a-9fca-30e6177a469b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:35.859078 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bb341-afc3-423a-9fca-30e6177a469b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:35.859407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bb341-afc3-423a-9fca-30e6177a469b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:35.860198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bb341-afc3-423a-9fca-30e6177a469b None admin] RBAC: Adding query filter params (role.id=9dc988eb162e47e48df810053eecb165, user.id=668fea2611474386a5d869d7778c84d9, scope.system=all) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:35.860198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bb341-afc3-423a-9fca-30e6177a469b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:35.862496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bb341-afc3-423a-9fca-30e6177a469b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:35.883458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 168/334] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Jul 25 18:42:35 2022] GET /identity/v3/role_assignments?user.id=668fea2611474386a5d869d7778c84d9&scope.system=all&role.id=9dc988eb162e47e48df810053eecb165 => generated 510 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:37.075279 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb08e4-8b46-4c67-bd13-7f5624314ad2 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.076095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb08e4-8b46-4c67-bd13-7f5624314ad2 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.076095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb08e4-8b46-4c67-bd13-7f5624314ad2 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:37.078170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 167/335] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:37.086829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8d41f-99bb-4a6c-b3dc-b4ac52a011bd None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.087579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8d41f-99bb-4a6c-b3dc-b4ac52a011bd None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.087579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8d41f-99bb-4a6c-b3dc-b4ac52a011bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:37.116427 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3fa8d41f-99bb-4a6c-b3dc-b4ac52a011bd None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:37.117711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3fa8d41f-99bb-4a6c-b3dc-b4ac52a011bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:37.233236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 169/336] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:37.274017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb08e4-8b46-4c67-bd13-7f5624314ad2 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:37.509430 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb08e4-8b46-4c67-bd13-7f5624314ad2 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:37.555064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2dda66-265b-4044-8dc0-b3da0043763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:37.555897 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dda66-265b-4044-8dc0-b3da0043763b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.556189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dda66-265b-4044-8dc0-b3da0043763b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.556474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dda66-265b-4044-8dc0-b3da0043763b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:37.573408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2dda66-265b-4044-8dc0-b3da0043763b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-fe2dda66-265b-4044-8dc0-b3da0043763b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:37.581691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:37.582806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:37.583734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:37.583734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:37.583734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:37.583734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:37.583734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:37.583734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:37.583734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:37.583734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:37.583734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:37.583734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:37.583734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:37.583734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:37.583734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 168/337] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:37.589234 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8d41f-99bb-4a6c-b3dc-b4ac52a011bd None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:37.630905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:37.728292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8d41f-99bb-4a6c-b3dc-b4ac52a011bd None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:37.773173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9c337-c997-4025-a182-2dc5fc55ec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:37.773773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9c337-c997-4025-a182-2dc5fc55ec44 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.774064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9c337-c997-4025-a182-2dc5fc55ec44 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.774386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9c337-c997-4025-a182-2dc5fc55ec44 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:37.775195 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9c337-c997-4025-a182-2dc5fc55ec44 None admin] RBAC: Adding query filter params (name=member) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:37.775511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9c337-c997-4025-a182-2dc5fc55ec44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:37.776906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9c337-c997-4025-a182-2dc5fc55ec44 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:37.790444 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 170/338] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:37.792835 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2dda66-265b-4044-8dc0-b3da0043763b None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:37.855491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:37.946704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2dda66-265b-4044-8dc0-b3da0043763b None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:37.993741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa4906-195b-4ab7-b870-44220cfa7a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:37.994462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa4906-195b-4ab7-b870-44220cfa7a89 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:37.994756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa4906-195b-4ab7-b870-44220cfa7a89 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:37.995038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa4906-195b-4ab7-b870-44220cfa7a89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:38.082957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa4906-195b-4ab7-b870-44220cfa7a89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:38.084420 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa4906-195b-4ab7-b870-44220cfa7a89 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:38.152730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:38.219745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 169/339] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:38.227506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9c337-c997-4025-a182-2dc5fc55ec44 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:38.385103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:38.411006 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9c337-c997-4025-a182-2dc5fc55ec44 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:38.463350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f13ba-69d5-4e96-8bea-64bd5ef53a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:38.464434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f13ba-69d5-4e96-8bea-64bd5ef53a1c None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:38.464776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f13ba-69d5-4e96-8bea-64bd5ef53a1c None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:38.465103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f13ba-69d5-4e96-8bea-64bd5ef53a1c None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:38.466904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f13ba-69d5-4e96-8bea-64bd5ef53a1c None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:38.468657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f13ba-69d5-4e96-8bea-64bd5ef53a1c None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:38.474446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 171/340] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:38 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:38.477853 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa4906-195b-4ab7-b870-44220cfa7a89 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:38.570134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80981c09-ec67-49d1-be8e-fdfe13b78b6f None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:38.570476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80981c09-ec67-49d1-be8e-fdfe13b78b6f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:38.570745 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80981c09-ec67-49d1-be8e-fdfe13b78b6f None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:38.572220 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 172/341] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:38.579859 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097c579-b797-4c3c-ab4a-1b2c7996683c None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:38.580383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097c579-b797-4c3c-ab4a-1b2c7996683c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:38.580825 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097c579-b797-4c3c-ab4a-1b2c7996683c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:38.609118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-f097c579-b797-4c3c-ab4a-1b2c7996683c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:38.610362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f097c579-b797-4c3c-ab4a-1b2c7996683c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:38.682256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa4906-195b-4ab7-b870-44220cfa7a89 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:38.763447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ba843-ebc5-4036-9ce7-d35a01d5ee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:38.764709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ba843-ebc5-4036-9ce7-d35a01d5ee1f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:38.765168 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 173/342] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:38.765669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ba843-ebc5-4036-9ce7-d35a01d5ee1f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:38.766175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ba843-ebc5-4036-9ce7-d35a01d5ee1f None admin] PATH_INFO: `/v3/groups/3e9ece46115d4d7caa86f3fff59281dc` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:38.778672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ba843-ebc5-4036-9ce7-d35a01d5ee1f None admin] RBAC: Authorizing `identity:get_group(group_id=3e9ece46115d4d7caa86f3fff59281dc)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:38.780590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ba843-ebc5-4036-9ce7-d35a01d5ee1f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:38.782907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 170/343] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:38 2022] GET /identity/v3/groups/3e9ece46115d4d7caa86f3fff59281dc => generated 226 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:38.789463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097c579-b797-4c3c-ab4a-1b2c7996683c None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:38.858580 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ba843-ebc5-4036-9ce7-d35a01d5ee1f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:38.888689 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:38.916587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:38.962649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097c579-b797-4c3c-ab4a-1b2c7996683c None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.014972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ba843-ebc5-4036-9ce7-d35a01d5ee1f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.023289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a831ff-a6c7-4748-88ba-5892d03a84f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.024070 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a831ff-a6c7-4748-88ba-5892d03a84f6 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.024412 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a831ff-a6c7-4748-88ba-5892d03a84f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.024746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a831ff-a6c7-4748-88ba-5892d03a84f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.025748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a831ff-a6c7-4748-88ba-5892d03a84f6 None admin] RBAC: Adding query filter params (group.id=3e9ece46115d4d7caa86f3fff59281dc, role.id=9dc988eb162e47e48df810053eecb165, scope.project.id=f0435acd015c454f926f9318bb5503b2) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:39.026097 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a831ff-a6c7-4748-88ba-5892d03a84f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:39.028603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a831ff-a6c7-4748-88ba-5892d03a84f6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:39.039030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 174/344] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 18:42:38 2022] GET /identity/v3/role_assignments?group.id=3e9ece46115d4d7caa86f3fff59281dc&scope.project.id=f0435acd015c454f926f9318bb5503b2&role.id=9dc988eb162e47e48df810053eecb165 => generated 260 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:39.040499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1901826-316d-49d4-ae72-f0418de33083 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.040816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1901826-316d-49d4-ae72-f0418de33083 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.041150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1901826-316d-49d4-ae72-f0418de33083 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.043042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 175/345] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:39.045621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2c370-d471-4365-8eab-11e56bdcb4cc None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.045961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2c370-d471-4365-8eab-11e56bdcb4cc None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.046280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2c370-d471-4365-8eab-11e56bdcb4cc None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.048713 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 176/346] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:39.050115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11823640-a287-4ddd-a627-884966c283c2 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.050450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11823640-a287-4ddd-a627-884966c283c2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.050763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11823640-a287-4ddd-a627-884966c283c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.079231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-11823640-a287-4ddd-a627-884966c283c2 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:39.080977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-11823640-a287-4ddd-a627-884966c283c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:39.102739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50cb01-6db9-453f-bbc5-2e064ee0325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.103598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50cb01-6db9-453f-bbc5-2e064ee0325a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.103892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50cb01-6db9-453f-bbc5-2e064ee0325a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.104171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50cb01-6db9-453f-bbc5-2e064ee0325a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.131809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50cb01-6db9-453f-bbc5-2e064ee0325a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-7f50cb01-6db9-453f-bbc5-2e064ee0325a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:39.146389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:39.147622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:39.148492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:39.148492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:39.148492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:39.148492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:39.148492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:39.148492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:39.148492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:39.148492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:39.148492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:39.148492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:39.148492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:39.148492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:39.148492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 171/347] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:39.149245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43716679-7752-47bb-a0d1-7dfee6c8796e None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.149245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43716679-7752-47bb-a0d1-7dfee6c8796e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.149446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43716679-7752-47bb-a0d1-7dfee6c8796e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.183392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-43716679-7752-47bb-a0d1-7dfee6c8796e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:39.185772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-43716679-7752-47bb-a0d1-7dfee6c8796e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:39.241611 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 177/348] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:39.245177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11823640-a287-4ddd-a627-884966c283c2 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:39.314472 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:39.323949 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 172/349] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:39.325065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43716679-7752-47bb-a0d1-7dfee6c8796e None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:39.391188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:39.431288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11823640-a287-4ddd-a627-884966c283c2 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.495372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43716679-7752-47bb-a0d1-7dfee6c8796e None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.500237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72feaeac-d7e6-466c-8eba-5111f994b073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.500431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72feaeac-d7e6-466c-8eba-5111f994b073 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.502081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72feaeac-d7e6-466c-8eba-5111f994b073 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.502081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72feaeac-d7e6-466c-8eba-5111f994b073 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.502081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72feaeac-d7e6-466c-8eba-5111f994b073 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:39.502407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72feaeac-d7e6-466c-8eba-5111f994b073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:39.504417 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72feaeac-d7e6-466c-8eba-5111f994b073 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:39.515024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 178/350] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:39.516319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72feaeac-d7e6-466c-8eba-5111f994b073 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:39.541458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a68c4-49c1-43a3-b16e-c27fb5ead989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.542662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a68c4-49c1-43a3-b16e-c27fb5ead989 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.543026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a68c4-49c1-43a3-b16e-c27fb5ead989 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.543380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a68c4-49c1-43a3-b16e-c27fb5ead989 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.558664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a68c4-49c1-43a3-b16e-c27fb5ead989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-893a68c4-49c1-43a3-b16e-c27fb5ead989 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:39.569218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:39.570261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:39.571226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:39.571226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:39.571226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:39.571226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:39.571226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:39.571226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:39.571226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:39.571226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:39.571226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:39.571226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:39.571226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:39.571226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:39.573634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 173/351] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:39.575029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665e926-8703-4cb0-98c9-932470043aff None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.575320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665e926-8703-4cb0-98c9-932470043aff None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.575605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665e926-8703-4cb0-98c9-932470043aff None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.576946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 174/352] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:39.578212 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b45d-fb66-4a70-809e-275351b02963 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.584057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b45d-fb66-4a70-809e-275351b02963 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.584749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b45d-fb66-4a70-809e-275351b02963 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.587514 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 175/353] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:39.589162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b45d-fb66-4a70-809e-275351b02963 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:39.660596 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72feaeac-d7e6-466c-8eba-5111f994b073 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.686159 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:39.703862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d411a-4c09-4ef8-aad9-640407e855f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.704679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d411a-4c09-4ef8-aad9-640407e855f4 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.704969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d411a-4c09-4ef8-aad9-640407e855f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.705254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d411a-4c09-4ef8-aad9-640407e855f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.717576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:39.720122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d411a-4c09-4ef8-aad9-640407e855f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-a97d411a-4c09-4ef8-aad9-640407e855f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:39.729227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:39.735113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 179/354] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:39.739350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d411a-4c09-4ef8-aad9-640407e855f4 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:39.770965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b45d-fb66-4a70-809e-275351b02963 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.828251 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103da2a-92d1-4beb-99bd-a32097790f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.829091 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103da2a-92d1-4beb-99bd-a32097790f5a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.829401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103da2a-92d1-4beb-99bd-a32097790f5a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.829681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103da2a-92d1-4beb-99bd-a32097790f5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.900710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d411a-4c09-4ef8-aad9-640407e855f4 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:39.942759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103da2a-92d1-4beb-99bd-a32097790f5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:39.945197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103da2a-92d1-4beb-99bd-a32097790f5a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:39.950142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db2319-27cb-4f66-93f2-027f9852b857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:39.950875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db2319-27cb-4f66-93f2-027f9852b857 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.951169 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db2319-27cb-4f66-93f2-027f9852b857 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.951450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db2319-27cb-4f66-93f2-027f9852b857 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.952211 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db2319-27cb-4f66-93f2-027f9852b857 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:39.952525 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db2319-27cb-4f66-93f2-027f9852b857 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:39.953946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db2319-27cb-4f66-93f2-027f9852b857 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:39.965636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 180/355] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:39.966975 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd100c-4575-4423-93d1-a4f6d1169820 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:39.967294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd100c-4575-4423-93d1-a4f6d1169820 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:39.967587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd100c-4575-4423-93d1-a4f6d1169820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:39.990881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-89bd100c-4575-4423-93d1-a4f6d1169820 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:39.992126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-89bd100c-4575-4423-93d1-a4f6d1169820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:40.023989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:40.081026 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:40.094261 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 181/356] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:40.095944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f9b54-c6cc-4436-9a1c-b29c00a54402 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:40.096388 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f9b54-c6cc-4436-9a1c-b29c00a54402 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:40.096839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f9b54-c6cc-4436-9a1c-b29c00a54402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:40.101575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 176/357] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:40.102533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103da2a-92d1-4beb-99bd-a32097790f5a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:40.118777 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-0a8f9b54-c6cc-4436-9a1c-b29c00a54402 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:40.120903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0a8f9b54-c6cc-4436-9a1c-b29c00a54402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:40.224734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 182/358] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:40.226134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f9b54-c6cc-4436-9a1c-b29c00a54402 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:40.254939 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103da2a-92d1-4beb-99bd-a32097790f5a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:40.305900 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf44c5-fe9a-4aa2-a77b-0529a8ba7fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:40.306547 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf44c5-fe9a-4aa2-a77b-0529a8ba7fa3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:40.306835 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf44c5-fe9a-4aa2-a77b-0529a8ba7fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:40.307115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf44c5-fe9a-4aa2-a77b-0529a8ba7fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:40.308396 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf44c5-fe9a-4aa2-a77b-0529a8ba7fa3 None admin] RBAC: Adding query filter params (name=member) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:40.308718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf44c5-fe9a-4aa2-a77b-0529a8ba7fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:40.310500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf44c5-fe9a-4aa2-a77b-0529a8ba7fa3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:40.323385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 177/359] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:40.324644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf44c5-fe9a-4aa2-a77b-0529a8ba7fa3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:40.355468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:40.369260 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f9b54-c6cc-4436-9a1c-b29c00a54402 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:40.402801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:40.422440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5267ac-ab69-4bff-b969-5645df268b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:40.423603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5267ac-ab69-4bff-b969-5645df268b90 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:40.424126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5267ac-ab69-4bff-b969-5645df268b90 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:40.424640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5267ac-ab69-4bff-b969-5645df268b90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:40.478470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf44c5-fe9a-4aa2-a77b-0529a8ba7fa3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:40.537701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf9eca-80d9-41ea-ba73-01a8f068725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:40.538877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf9eca-80d9-41ea-ba73-01a8f068725b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:40.539353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf9eca-80d9-41ea-ba73-01a8f068725b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:40.539807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf9eca-80d9-41ea-ba73-01a8f068725b None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:40.541272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf9eca-80d9-41ea-ba73-01a8f068725b None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:40.542769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf9eca-80d9-41ea-ba73-01a8f068725b None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:40.545588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 178/360] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:40 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:40.546258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5267ac-ab69-4bff-b969-5645df268b90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:40.546738 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf9eca-80d9-41ea-ba73-01a8f068725b None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:40.548407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5267ac-ab69-4bff-b969-5645df268b90 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:40.685143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:40.698131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf9eca-80d9-41ea-ba73-01a8f068725b None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:40.705667 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 183/361] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:40.706873 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5267ac-ab69-4bff-b969-5645df268b90 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:40.742276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:40.746508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab43735-c17c-435a-9e0b-f5a6193f85c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:40.747332 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab43735-c17c-435a-9e0b-f5a6193f85c3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:40.747619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab43735-c17c-435a-9e0b-f5a6193f85c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:40.747892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab43735-c17c-435a-9e0b-f5a6193f85c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:40.761184 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab43735-c17c-435a-9e0b-f5a6193f85c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-aab43735-c17c-435a-9e0b-f5a6193f85c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:40.769571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:40.771644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:40.772505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:40.772505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:40.772505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:40.772505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:40.772505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:40.772505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:40.772505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:40.772505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:40.772505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:40.772505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:40.772505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:40.772505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:40.775848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 179/362] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:40.782015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab43735-c17c-435a-9e0b-f5a6193f85c3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:40.873207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5267ac-ab69-4bff-b969-5645df268b90 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:40.932206 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cc700-0566-4575-95f2-4527f05cdb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:40.933041 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cc700-0566-4575-95f2-4527f05cdb96 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:40.933333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cc700-0566-4575-95f2-4527f05cdb96 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:40.933843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cc700-0566-4575-95f2-4527f05cdb96 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:40.944190 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab43735-c17c-435a-9e0b-f5a6193f85c3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:40.952339 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cc700-0566-4575-95f2-4527f05cdb96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-7c2cc700-0566-4575-95f2-4527f05cdb96 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:40.960573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:40.961517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:40.962316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:40.962316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:40.962316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:40.962316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:40.962316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:40.962316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:40.962316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:40.962316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:40.962316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:40.962316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:40.962316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 18:42:40.962316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:40.962316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 184/363] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:40.967787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cc700-0566-4575-95f2-4527f05cdb96 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:40.998998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f30e7d-f2e4-49b3-92c9-4a6a652f7822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:40.999694 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f30e7d-f2e4-49b3-92c9-4a6a652f7822 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:40.999983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f30e7d-f2e4-49b3-92c9-4a6a652f7822 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.000266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f30e7d-f2e4-49b3-92c9-4a6a652f7822 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.025940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:41.102371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f30e7d-f2e4-49b3-92c9-4a6a652f7822 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:41.104432 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f30e7d-f2e4-49b3-92c9-4a6a652f7822 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:41.120777 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:41.163383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cc700-0566-4575-95f2-4527f05cdb96 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:41.226565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a1603-948a-48c8-829c-c7e41fe29f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:41.227505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a1603-948a-48c8-829c-c7e41fe29f7b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.227804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a1603-948a-48c8-829c-c7e41fe29f7b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.228092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a1603-948a-48c8-829c-c7e41fe29f7b None admin] PATH_INFO: `/v3/groups/3e9ece46115d4d7caa86f3fff59281dc` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.235596 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a1603-948a-48c8-829c-c7e41fe29f7b None admin] RBAC: Authorizing `identity:get_group(group_id=3e9ece46115d4d7caa86f3fff59281dc)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:41.235596 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a1603-948a-48c8-829c-c7e41fe29f7b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:41.235596 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 185/364] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:40 2022] GET /identity/v3/groups/3e9ece46115d4d7caa86f3fff59281dc => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:41.236258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a1603-948a-48c8-829c-c7e41fe29f7b None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:41.264628 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 180/365] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:41.265513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f30e7d-f2e4-49b3-92c9-4a6a652f7822 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:41.336507 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:41.418470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f30e7d-f2e4-49b3-92c9-4a6a652f7822 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:41.420505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a1603-948a-48c8-829c-c7e41fe29f7b None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:41.485731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56708611-5425-42b1-b1ad-c8086eb5f65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:41.486561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae9773-fcab-4ec3-8527-58a2ee7d7564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:41.487563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56708611-5425-42b1-b1ad-c8086eb5f65f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.488061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae9773-fcab-4ec3-8527-58a2ee7d7564 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.488191 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56708611-5425-42b1-b1ad-c8086eb5f65f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.488676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae9773-fcab-4ec3-8527-58a2ee7d7564 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.488911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56708611-5425-42b1-b1ad-c8086eb5f65f None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.489414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae9773-fcab-4ec3-8527-58a2ee7d7564 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.490658 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56708611-5425-42b1-b1ad-c8086eb5f65f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:41.491373 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae9773-fcab-4ec3-8527-58a2ee7d7564 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:41.491704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56708611-5425-42b1-b1ad-c8086eb5f65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:41.493237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae9773-fcab-4ec3-8527-58a2ee7d7564 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:41.494147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56708611-5425-42b1-b1ad-c8086eb5f65f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:41.501731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 186/366] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:41 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:41.503253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae9773-fcab-4ec3-8527-58a2ee7d7564 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:41.513081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 181/367] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:41.514830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56708611-5425-42b1-b1ad-c8086eb5f65f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:41.535808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:41.646789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae9773-fcab-4ec3-8527-58a2ee7d7564 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:41.667830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56708611-5425-42b1-b1ad-c8086eb5f65f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:41.683231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:41.689782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103fd4c3-7832-4b41-a816-9d75646066cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:41.690655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fd4c3-7832-4b41-a816-9d75646066cb None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.691048 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fd4c3-7832-4b41-a816-9d75646066cb None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.691351 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fd4c3-7832-4b41-a816-9d75646066cb None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.692287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103fd4c3-7832-4b41-a816-9d75646066cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:41.692599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103fd4c3-7832-4b41-a816-9d75646066cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:41.695185 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103fd4c3-7832-4b41-a816-9d75646066cb None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:41.705177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 187/368] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:41.706580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103fd4c3-7832-4b41-a816-9d75646066cb None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:41.706695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d670e-ed94-48b9-96e9-6f7fe7b94e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:41.707179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d670e-ed94-48b9-96e9-6f7fe7b94e34 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.707531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d670e-ed94-48b9-96e9-6f7fe7b94e34 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.707843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d670e-ed94-48b9-96e9-6f7fe7b94e34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.708909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d670e-ed94-48b9-96e9-6f7fe7b94e34 None admin] RBAC: Adding query filter params (group.id=3e9ece46115d4d7caa86f3fff59281dc, role.id=a638e91b70624f669ffe76dac32323f5, scope.project.id=f0435acd015c454f926f9318bb5503b2) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:41.709450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d670e-ed94-48b9-96e9-6f7fe7b94e34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:41.711215 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d670e-ed94-48b9-96e9-6f7fe7b94e34 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:41.718385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 182/369] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 18:42:41 2022] GET /identity/v3/role_assignments?group.id=3e9ece46115d4d7caa86f3fff59281dc&scope.project.id=f0435acd015c454f926f9318bb5503b2&role.id=a638e91b70624f669ffe76dac32323f5 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:41.719193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d670e-ed94-48b9-96e9-6f7fe7b94e34 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:41.834150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:41.860396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103fd4c3-7832-4b41-a816-9d75646066cb None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:41.887286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d670e-ed94-48b9-96e9-6f7fe7b94e34 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:41.917221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc716afa-0c6d-44dd-ae90-8f4a0f71ceb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:41.918302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc716afa-0c6d-44dd-ae90-8f4a0f71ceb2 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.918804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc716afa-0c6d-44dd-ae90-8f4a0f71ceb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.919139 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc716afa-0c6d-44dd-ae90-8f4a0f71ceb2 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.920716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc716afa-0c6d-44dd-ae90-8f4a0f71ceb2 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:41.922766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc716afa-0c6d-44dd-ae90-8f4a0f71ceb2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:41.932723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 188/370] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:41 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:41.932723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc716afa-0c6d-44dd-ae90-8f4a0f71ceb2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:41.948049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e61ca-bc1f-4456-bf0f-b5dbfff693de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:41.949089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e61ca-bc1f-4456-bf0f-b5dbfff693de None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.949378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e61ca-bc1f-4456-bf0f-b5dbfff693de None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.949688 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e61ca-bc1f-4456-bf0f-b5dbfff693de None admin] PATH_INFO: `/v3/groups/3e9ece46115d4d7caa86f3fff59281dc` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.951089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e61ca-bc1f-4456-bf0f-b5dbfff693de None admin] RBAC: Authorizing `identity:get_group(group_id=3e9ece46115d4d7caa86f3fff59281dc)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:41.952788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e61ca-bc1f-4456-bf0f-b5dbfff693de None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:41.954655 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 183/371] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:41 2022] GET /identity/v3/groups/3e9ece46115d4d7caa86f3fff59281dc => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:41.961849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515dcf7-6203-40a4-a3af-4f6d6a41fd87 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:41.962146 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515dcf7-6203-40a4-a3af-4f6d6a41fd87 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:41.962457 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515dcf7-6203-40a4-a3af-4f6d6a41fd87 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:41.964349 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 184/372] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:41.965243 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515dcf7-6203-40a4-a3af-4f6d6a41fd87 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.039040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:42.064835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc716afa-0c6d-44dd-ae90-8f4a0f71ceb2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.110207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515dcf7-6203-40a4-a3af-4f6d6a41fd87 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.110681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ef99d-d279-403a-8b2f-1520e87baab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.111364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ef99d-d279-403a-8b2f-1520e87baab7 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.111759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ef99d-d279-403a-8b2f-1520e87baab7 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.112002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ef99d-d279-403a-8b2f-1520e87baab7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.135320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:42.165344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41a992-4d02-477b-8860-e5355a73f942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.166665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41a992-4d02-477b-8860-e5355a73f942 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.167001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41a992-4d02-477b-8860-e5355a73f942 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.167296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41a992-4d02-477b-8860-e5355a73f942 None admin] PATH_INFO: `/v3/users/0726b1b313114751a97d0f7ab41f8803` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.168876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41a992-4d02-477b-8860-e5355a73f942 None admin] RBAC: Authorizing `identity:get_user(user_id=0726b1b313114751a97d0f7ab41f8803)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:42.170411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41a992-4d02-477b-8860-e5355a73f942 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:42.172101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 185/373] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:42:41 2022] GET /identity/v3/users/0726b1b313114751a97d0f7ab41f8803 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:42.172880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41a992-4d02-477b-8860-e5355a73f942 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.207011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ef99d-d279-403a-8b2f-1520e87baab7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:42.209098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ef99d-d279-403a-8b2f-1520e87baab7 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:42.297245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41a992-4d02-477b-8860-e5355a73f942 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.313578 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:42.337911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fa95c-c205-4432-91d6-475c2ff943ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.341810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fa95c-c205-4432-91d6-475c2ff943ce None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.341810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fa95c-c205-4432-91d6-475c2ff943ce None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.341810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fa95c-c205-4432-91d6-475c2ff943ce None admin] PATH_INFO: `/v3/groups/3e9ece46115d4d7caa86f3fff59281dc` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.347147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 189/374] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:42.347147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ef99d-d279-403a-8b2f-1520e87baab7 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.348163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fa95c-c205-4432-91d6-475c2ff943ce None admin] RBAC: Authorizing `identity:get_group(group_id=3e9ece46115d4d7caa86f3fff59281dc)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:42.350070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fa95c-c205-4432-91d6-475c2ff943ce None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:42.357666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 186/375] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/groups/3e9ece46115d4d7caa86f3fff59281dc => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:42.357666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a813e-ac90-4ad7-93e3-6e43546bdbd1 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.357666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a813e-ac90-4ad7-93e3-6e43546bdbd1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.357666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a813e-ac90-4ad7-93e3-6e43546bdbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.375142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d65a813e-ac90-4ad7-93e3-6e43546bdbd1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:42.376631 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d65a813e-ac90-4ad7-93e3-6e43546bdbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:42.407910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:42.476563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 187/376] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:42.480999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a813e-ac90-4ad7-93e3-6e43546bdbd1 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.492033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ef99d-d279-403a-8b2f-1520e87baab7 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.546312 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cdbc9-3541-4c3e-8990-38c60c5cf7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.547292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cdbc9-3541-4c3e-8990-38c60c5cf7be None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.547292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cdbc9-3541-4c3e-8990-38c60c5cf7be None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.547643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cdbc9-3541-4c3e-8990-38c60c5cf7be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.548416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cdbc9-3541-4c3e-8990-38c60c5cf7be None admin] RBAC: Adding query filter params (group.id=3e9ece46115d4d7caa86f3fff59281dc, role.id=a638e91b70624f669ffe76dac32323f5, scope.project.id=8cea8011767244e0a581010033a1caf4) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:42.548733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cdbc9-3541-4c3e-8990-38c60c5cf7be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:42.551900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cdbc9-3541-4c3e-8990-38c60c5cf7be None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:42.557639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 190/377] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/role_assignments?group.id=3e9ece46115d4d7caa86f3fff59281dc&scope.project.id=8cea8011767244e0a581010033a1caf4&role.id=a638e91b70624f669ffe76dac32323f5 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:42.564078 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cdbc9-3541-4c3e-8990-38c60c5cf7be None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.642106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a813e-ac90-4ad7-93e3-6e43546bdbd1 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.657516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:42.692630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76d38e-61e2-4f2b-9bbb-5c8e06366e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.694179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76d38e-61e2-4f2b-9bbb-5c8e06366e77 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.694743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76d38e-61e2-4f2b-9bbb-5c8e06366e77 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.695233 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76d38e-61e2-4f2b-9bbb-5c8e06366e77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.696415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76d38e-61e2-4f2b-9bbb-5c8e06366e77 None admin] RBAC: Adding query filter params (role.id=b2f299f9533944ca821a102e13bd4c69, user.id=0726b1b313114751a97d0f7ab41f8803, scope.system=all) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:42.696984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76d38e-61e2-4f2b-9bbb-5c8e06366e77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:42.699003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76d38e-61e2-4f2b-9bbb-5c8e06366e77 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:42.723778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 188/378] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/role_assignments?user.id=0726b1b313114751a97d0f7ab41f8803&scope.system=all&role.id=b2f299f9533944ca821a102e13bd4c69 => generated 510 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:42.726124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76d38e-61e2-4f2b-9bbb-5c8e06366e77 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.743716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cdbc9-3541-4c3e-8990-38c60c5cf7be None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.779764 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:42.794900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f3da9-7143-4958-9a72-d0a9fbae9098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.795969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f3da9-7143-4958-9a72-d0a9fbae9098 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.796264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f3da9-7143-4958-9a72-d0a9fbae9098 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.796651 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f3da9-7143-4958-9a72-d0a9fbae9098 None admin] PATH_INFO: `/v3/projects/c39adca8a83c49e4a2ab45e68b3a9724` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.798203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f3da9-7143-4958-9a72-d0a9fbae9098 None admin] RBAC: Authorizing `identity:get_project(project_id=c39adca8a83c49e4a2ab45e68b3a9724)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:42.807609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f3da9-7143-4958-9a72-d0a9fbae9098 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:42.810031 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 191/379] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/projects/c39adca8a83c49e4a2ab45e68b3a9724 => generated 344 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:42.810816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f3da9-7143-4958-9a72-d0a9fbae9098 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.878304 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76d38e-61e2-4f2b-9bbb-5c8e06366e77 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:42.965569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ab3ed-1ddb-4656-8e77-e9b223c81e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:42.966503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ab3ed-1ddb-4656-8e77-e9b223c81e58 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:42.966801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ab3ed-1ddb-4656-8e77-e9b223c81e58 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:42.967080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ab3ed-1ddb-4656-8e77-e9b223c81e58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:42.967933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ab3ed-1ddb-4656-8e77-e9b223c81e58 None admin] RBAC: Adding query filter params (group.id=3e9ece46115d4d7caa86f3fff59281dc, role.id=9dc988eb162e47e48df810053eecb165, scope.project.id=8cea8011767244e0a581010033a1caf4) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:42.968250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ab3ed-1ddb-4656-8e77-e9b223c81e58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:42.970527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ab3ed-1ddb-4656-8e77-e9b223c81e58 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:42.980286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 189/380] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/role_assignments?group.id=3e9ece46115d4d7caa86f3fff59281dc&scope.project.id=8cea8011767244e0a581010033a1caf4&role.id=9dc988eb162e47e48df810053eecb165 => generated 260 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:42.981162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ab3ed-1ddb-4656-8e77-e9b223c81e58 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:42.992870 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f3da9-7143-4958-9a72-d0a9fbae9098 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:43.040108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:43.049921 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9273a2-8df2-4d2a-b92f-22f3b74f7046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:43.052200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9273a2-8df2-4d2a-b92f-22f3b74f7046 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:43.052750 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9273a2-8df2-4d2a-b92f-22f3b74f7046 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:43.053240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9273a2-8df2-4d2a-b92f-22f3b74f7046 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:43.077315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9273a2-8df2-4d2a-b92f-22f3b74f7046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-ae9273a2-8df2-4d2a-b92f-22f3b74f7046 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:43.095090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:43.096093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:43.096972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:43.096972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:43.096972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:43.096972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:43.096972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:43.096972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:43.096972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:43.096972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:43.096972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:43.096972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:43.096972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:43.096972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:43.106466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 192/381] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:43.107817 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9273a2-8df2-4d2a-b92f-22f3b74f7046 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:43.202225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:43.228459 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ab3ed-1ddb-4656-8e77-e9b223c81e58 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:43.311162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e1558-9886-4d98-af33-233840e09c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:43.313418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e1558-9886-4d98-af33-233840e09c98 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:43.313895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e1558-9886-4d98-af33-233840e09c98 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:43.314413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e1558-9886-4d98-af33-233840e09c98 None admin] PATH_INFO: `/v3/groups/8aee740498a84f17bb981120ce74d892` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:43.324928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e1558-9886-4d98-af33-233840e09c98 None admin] RBAC: Authorizing `identity:get_group(group_id=8aee740498a84f17bb981120ce74d892)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:43.328305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e1558-9886-4d98-af33-233840e09c98 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:43.332261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 190/382] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:42 2022] GET /identity/v3/groups/8aee740498a84f17bb981120ce74d892 => generated 229 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:43.336171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9273a2-8df2-4d2a-b92f-22f3b74f7046 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:43.343436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e1558-9886-4d98-af33-233840e09c98 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:43.388097 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543af1b-0f4a-4236-9c4c-82bcf2fc20a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:43.388914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543af1b-0f4a-4236-9c4c-82bcf2fc20a6 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:43.389261 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543af1b-0f4a-4236-9c4c-82bcf2fc20a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:43.389554 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543af1b-0f4a-4236-9c4c-82bcf2fc20a6 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:43.391518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543af1b-0f4a-4236-9c4c-82bcf2fc20a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:43.391941 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543af1b-0f4a-4236-9c4c-82bcf2fc20a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:43.394488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543af1b-0f4a-4236-9c4c-82bcf2fc20a6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:43.412751 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 193/383] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:43.418308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543af1b-0f4a-4236-9c4c-82bcf2fc20a6 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:43.568493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:43.578564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e1558-9886-4d98-af33-233840e09c98 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:43.635434 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07495c-c9cb-4a86-9b42-3f6c40b9843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:43.637895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07495c-c9cb-4a86-9b42-3f6c40b9843c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:43.638423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07495c-c9cb-4a86-9b42-3f6c40b9843c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:43.639112 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07495c-c9cb-4a86-9b42-3f6c40b9843c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:43.641913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07495c-c9cb-4a86-9b42-3f6c40b9843c None admin] RBAC: Adding query filter params (group.id=8aee740498a84f17bb981120ce74d892, role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=c39adca8a83c49e4a2ab45e68b3a9724) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:43.644745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07495c-c9cb-4a86-9b42-3f6c40b9843c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:43.647273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07495c-c9cb-4a86-9b42-3f6c40b9843c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:43.665003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 191/384] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 18:42:43 2022] GET /identity/v3/role_assignments?group.id=8aee740498a84f17bb981120ce74d892&scope.project.id=c39adca8a83c49e4a2ab45e68b3a9724&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 260 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:43.674656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543af1b-0f4a-4236-9c4c-82bcf2fc20a6 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:43.677273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:43.719735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29479b-1724-4f29-88f4-4440787f2200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:43.720969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29479b-1724-4f29-88f4-4440787f2200 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:43.721449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29479b-1724-4f29-88f4-4440787f2200 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:43.721922 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29479b-1724-4f29-88f4-4440787f2200 None admin] PATH_INFO: `/v3/groups/3e9ece46115d4d7caa86f3fff59281dc` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:43.723903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29479b-1724-4f29-88f4-4440787f2200 None admin] RBAC: Authorizing `identity:get_group(group_id=3e9ece46115d4d7caa86f3fff59281dc)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:43.727270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29479b-1724-4f29-88f4-4440787f2200 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:43.729413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 194/385] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:43 2022] GET /identity/v3/groups/3e9ece46115d4d7caa86f3fff59281dc => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:43.736221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07495c-c9cb-4a86-9b42-3f6c40b9843c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:43.894837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07495c-c9cb-4a86-9b42-3f6c40b9843c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:43.936071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ebb90-550e-4ce3-802e-8c5093e0a76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:43.936970 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ebb90-550e-4ce3-802e-8c5093e0a76c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:43.937258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ebb90-550e-4ce3-802e-8c5093e0a76c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:43.937536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ebb90-550e-4ce3-802e-8c5093e0a76c None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:43.938861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ebb90-550e-4ce3-802e-8c5093e0a76c None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:43.940477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ebb90-550e-4ce3-802e-8c5093e0a76c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:43.942546 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 192/386] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:43 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:43.948507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29479b-1724-4f29-88f4-4440787f2200 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:44.104791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29479b-1724-4f29-88f4-4440787f2200 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:44.154063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eba547-ff3f-45d4-aeed-d1e28b3428a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:44.154929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eba547-ff3f-45d4-aeed-d1e28b3428a4 None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:44.155216 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eba547-ff3f-45d4-aeed-d1e28b3428a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:44.155500 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eba547-ff3f-45d4-aeed-d1e28b3428a4 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2/groups/3e9ece46115d4d7caa86f3fff59281dc/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:44.175539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eba547-ff3f-45d4-aeed-d1e28b3428a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0435acd015c454f926f9318bb5503b2, group_id=3e9ece46115d4d7caa86f3fff59281dc, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:44.175539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eba547-ff3f-45d4-aeed-d1e28b3428a4 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:44.247946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a886e-5f5f-4768-a0d9-fe86f4e76993 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:44.248277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a886e-5f5f-4768-a0d9-fe86f4e76993 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:44.248511 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a886e-5f5f-4768-a0d9-fe86f4e76993 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:44.250146 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 193/387] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:44.256361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4947735-f194-48f4-b96e-ef8cd0a75621 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:44.256863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4947735-f194-48f4-b96e-ef8cd0a75621 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:44.257244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4947735-f194-48f4-b96e-ef8cd0a75621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:44.279500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-e4947735-f194-48f4-b96e-ef8cd0a75621 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:44.281044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e4947735-f194-48f4-b96e-ef8cd0a75621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:44.309320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:44.336835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 195/388] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Jul 25 18:42:43 2022] PUT /identity/v3/projects/f0435acd015c454f926f9318bb5503b2/groups/3e9ece46115d4d7caa86f3fff59281dc/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:44.364558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 194/389] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:44.404157 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eba547-ff3f-45d4-aeed-d1e28b3428a4 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:44.456449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:44.564015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eba547-ff3f-45d4-aeed-d1e28b3428a4 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:44.606174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40688dd6-b304-419d-bc7e-3206ed6caf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:44.607065 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40688dd6-b304-419d-bc7e-3206ed6caf81 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:44.607352 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40688dd6-b304-419d-bc7e-3206ed6caf81 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:44.607628 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40688dd6-b304-419d-bc7e-3206ed6caf81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:44.620957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40688dd6-b304-419d-bc7e-3206ed6caf81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-40688dd6-b304-419d-bc7e-3206ed6caf81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:44.629649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:44.630709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:44.631547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:44.631547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:44.631547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:44.631547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:44.631547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:44.631547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:44.631547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:44.631547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:44.631547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:44.631547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:44.631547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:44.631547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:44.631547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 196/390] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:44.637566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4947735-f194-48f4-b96e-ef8cd0a75621 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:44.770313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4947735-f194-48f4-b96e-ef8cd0a75621 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:44.827196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6991c7-d28a-4f43-bb94-f1c4e1fb9fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:44.827890 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6991c7-d28a-4f43-bb94-f1c4e1fb9fbe None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:44.828174 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6991c7-d28a-4f43-bb94-f1c4e1fb9fbe None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:44.828452 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6991c7-d28a-4f43-bb94-f1c4e1fb9fbe None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:44.829212 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6991c7-d28a-4f43-bb94-f1c4e1fb9fbe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:44.829522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6991c7-d28a-4f43-bb94-f1c4e1fb9fbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:44.830928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6991c7-d28a-4f43-bb94-f1c4e1fb9fbe None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:44.843494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 195/391] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:44.849877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40688dd6-b304-419d-bc7e-3206ed6caf81 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:44.978151 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:44.995514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40688dd6-b304-419d-bc7e-3206ed6caf81 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:45.064250 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1389b99-4002-4278-800a-9ee26c3cd6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:45.065551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389b99-4002-4278-800a-9ee26c3cd6aa None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.066065 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389b99-4002-4278-800a-9ee26c3cd6aa None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.066574 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389b99-4002-4278-800a-9ee26c3cd6aa None admin] PATH_INFO: `/v3/groups/3e9ece46115d4d7caa86f3fff59281dc` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.068891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1389b99-4002-4278-800a-9ee26c3cd6aa None admin] RBAC: Authorizing `identity:get_group(group_id=3e9ece46115d4d7caa86f3fff59281dc)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:45.071814 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1389b99-4002-4278-800a-9ee26c3cd6aa None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:45.073874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 197/392] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:44 2022] GET /identity/v3/groups/3e9ece46115d4d7caa86f3fff59281dc => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:45.079989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6991c7-d28a-4f43-bb94-f1c4e1fb9fbe None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:45.129400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:45.264890 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6991c7-d28a-4f43-bb94-f1c4e1fb9fbe None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:45.322348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40751d4-3068-424b-9b2f-fe7d6e372210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:45.323185 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40751d4-3068-424b-9b2f-fe7d6e372210 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.323475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40751d4-3068-424b-9b2f-fe7d6e372210 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.323760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40751d4-3068-424b-9b2f-fe7d6e372210 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.325040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40751d4-3068-424b-9b2f-fe7d6e372210 None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:45.326686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40751d4-3068-424b-9b2f-fe7d6e372210 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:45.328477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 196/393] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:45 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:45.334642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1389b99-4002-4278-800a-9ee26c3cd6aa None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:45.499944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1389b99-4002-4278-800a-9ee26c3cd6aa None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:45.542132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4f6b1-3a6c-42be-82f2-fed40ee67bec None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.542753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4f6b1-3a6c-42be-82f2-fed40ee67bec None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.542753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4f6b1-3a6c-42be-82f2-fed40ee67bec None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.544708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 197/394] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:45.547687 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d7181-6605-42c9-a63a-5aa8913d96eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:45.548631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d7181-6605-42c9-a63a-5aa8913d96eb None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.548921 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d7181-6605-42c9-a63a-5aa8913d96eb None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.549269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d7181-6605-42c9-a63a-5aa8913d96eb None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2/groups/3e9ece46115d4d7caa86f3fff59281dc/roles/a638e91b70624f669ffe76dac32323f5` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.551602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77184ac6-e312-41e9-848c-799bf5a4813f None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.552136 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77184ac6-e312-41e9-848c-799bf5a4813f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.553397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77184ac6-e312-41e9-848c-799bf5a4813f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.556423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d7181-6605-42c9-a63a-5aa8913d96eb None admin] RBAC: Authorizing `identity:create_grant(project_id=f0435acd015c454f926f9318bb5503b2, group_id=3e9ece46115d4d7caa86f3fff59281dc, role_id=a638e91b70624f669ffe76dac32323f5)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:45.562566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d7181-6605-42c9-a63a-5aa8913d96eb None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:45.584738 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-77184ac6-e312-41e9-848c-799bf5a4813f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:45.586531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-77184ac6-e312-41e9-848c-799bf5a4813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:45.640793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 198/395] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Jul 25 18:42:45 2022] PUT /identity/v3/projects/f0435acd015c454f926f9318bb5503b2/groups/3e9ece46115d4d7caa86f3fff59281dc/roles/a638e91b70624f669ffe76dac32323f5 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:45.725031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 198/396] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:45.763951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d7181-6605-42c9-a63a-5aa8913d96eb None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:45.806744 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:45.898145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708addc2-9c5d-49a9-8d25-911d2164460a None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.898420 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708addc2-9c5d-49a9-8d25-911d2164460a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.898655 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708addc2-9c5d-49a9-8d25-911d2164460a None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.903340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 199/397] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:45.906852 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca26e3-8709-457e-bf36-6d761c01b4a1 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.907212 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca26e3-8709-457e-bf36-6d761c01b4a1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.907511 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca26e3-8709-457e-bf36-6d761c01b4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:45.924593 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d7181-6605-42c9-a63a-5aa8913d96eb None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:45.933840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-55ca26e3-8709-457e-bf36-6d761c01b4a1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:45.935109 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-55ca26e3-8709-457e-bf36-6d761c01b4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:45.951950 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:45.983583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc11cd-5c36-46dc-9675-f809f1c8e1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:45.984414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc11cd-5c36-46dc-9675-f809f1c8e1ba None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:45.984704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc11cd-5c36-46dc-9675-f809f1c8e1ba None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:45.984986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc11cd-5c36-46dc-9675-f809f1c8e1ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.002252 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc11cd-5c36-46dc-9675-f809f1c8e1ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-11fc11cd-5c36-46dc-9675-f809f1c8e1ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:46.015110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:46.016074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:46.016904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:46.016904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:46.016904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:46.016904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:46.016904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:46.016904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:46.016904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:46.016904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:46.016904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:46.016904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:46.016904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:46.016904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:46.018198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 199/398] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:46.028031 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc11cd-5c36-46dc-9675-f809f1c8e1ba None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.090656 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 200/399] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:46.121467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca26e3-8709-457e-bf36-6d761c01b4a1 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.226012 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:46.257044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc11cd-5c36-46dc-9675-f809f1c8e1ba None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:46.344124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45227295-1186-47ed-8264-594fd0803d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:46.344900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45227295-1186-47ed-8264-594fd0803d2c None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.345258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45227295-1186-47ed-8264-594fd0803d2c None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.345577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45227295-1186-47ed-8264-594fd0803d2c None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.346897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45227295-1186-47ed-8264-594fd0803d2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:46.347222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45227295-1186-47ed-8264-594fd0803d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:46.349307 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45227295-1186-47ed-8264-594fd0803d2c None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:46.359838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 200/400] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:46.365430 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45227295-1186-47ed-8264-594fd0803d2c None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.412463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca26e3-8709-457e-bf36-6d761c01b4a1 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:46.483287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbb71d-a36a-4d80-b7ed-f13bef6f22b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:46.484570 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbb71d-a36a-4d80-b7ed-f13bef6f22b5 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.484883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbb71d-a36a-4d80-b7ed-f13bef6f22b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.485173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbb71d-a36a-4d80-b7ed-f13bef6f22b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.500673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbb71d-a36a-4d80-b7ed-f13bef6f22b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-1dbbb71d-a36a-4d80-b7ed-f13bef6f22b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:46.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:46.513808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:46.514636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:46.514636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:46.514636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:46.514636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:46.514636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:46.514636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:46.514636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:46.514636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:46.514636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:46.514636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:46.514636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:46.514636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:46.518438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 201/401] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:42:46.520268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b096776-3aa0-4e2b-bb30-0af9e4270c6a None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.521106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b096776-3aa0-4e2b-bb30-0af9e4270c6a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.521596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b096776-3aa0-4e2b-bb30-0af9e4270c6a None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.529824 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 202/402] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:46.532893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b096776-3aa0-4e2b-bb30-0af9e4270c6a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.546682 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45227295-1186-47ed-8264-594fd0803d2c None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:46.548257 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:46.588762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3aa91-24be-4b73-9674-e204b48a2498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:46.589522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3aa91-24be-4b73-9674-e204b48a2498 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.589868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3aa91-24be-4b73-9674-e204b48a2498 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.590159 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3aa91-24be-4b73-9674-e204b48a2498 None admin] PATH_INFO: `/v3/groups/3e9ece46115d4d7caa86f3fff59281dc` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.591493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3aa91-24be-4b73-9674-e204b48a2498 None admin] RBAC: Authorizing `identity:get_group(group_id=3e9ece46115d4d7caa86f3fff59281dc)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:46.593064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3aa91-24be-4b73-9674-e204b48a2498 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:46.596396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 201/403] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/groups/3e9ece46115d4d7caa86f3fff59281dc => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:46.598149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3d305-b75d-4f51-a10c-4794d3d21261 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.598475 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3d305-b75d-4f51-a10c-4794d3d21261 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.598875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3d305-b75d-4f51-a10c-4794d3d21261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.621180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-2ac3d305-b75d-4f51-a10c-4794d3d21261 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:46.622758 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2ac3d305-b75d-4f51-a10c-4794d3d21261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:46.625169 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:46.666998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b096776-3aa0-4e2b-bb30-0af9e4270c6a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:46.717906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b49c8-4744-4723-9d28-c27e7725c51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:46.721244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b49c8-4744-4723-9d28-c27e7725c51f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.722065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b49c8-4744-4723-9d28-c27e7725c51f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.722487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 202/404] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:46.723454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b49c8-4744-4723-9d28-c27e7725c51f None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.723739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3d305-b75d-4f51-a10c-4794d3d21261 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.725877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b49c8-4744-4723-9d28-c27e7725c51f None admin] RBAC: Adding query filter params (name=member) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:46.726483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b49c8-4744-4723-9d28-c27e7725c51f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:46.728179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b49c8-4744-4723-9d28-c27e7725c51f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:46.740015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 203/405] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:46.760763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b49c8-4744-4723-9d28-c27e7725c51f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.860646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3d305-b75d-4f51-a10c-4794d3d21261 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:46.868089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:46.913645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b49c8-4744-4723-9d28-c27e7725c51f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:46.914814 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64068e-6ff9-4ff4-bb19-768c2fed1c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:46.915710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64068e-6ff9-4ff4-bb19-768c2fed1c6f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.915998 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64068e-6ff9-4ff4-bb19-768c2fed1c6f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.916278 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64068e-6ff9-4ff4-bb19-768c2fed1c6f None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.917575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64068e-6ff9-4ff4-bb19-768c2fed1c6f None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:46.919757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64068e-6ff9-4ff4-bb19-768c2fed1c6f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:46.923145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 203/406] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:46.924093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64068e-6ff9-4ff4-bb19-768c2fed1c6f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:46.935049 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:46.960584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72057d-78e2-4db4-8a0e-dd1fee10f9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:46.961976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72057d-78e2-4db4-8a0e-dd1fee10f9f0 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:46.962627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72057d-78e2-4db4-8a0e-dd1fee10f9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:46.963135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72057d-78e2-4db4-8a0e-dd1fee10f9f0 None admin] PATH_INFO: `/v3/groups/3e9ece46115d4d7caa86f3fff59281dc` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:46.964810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72057d-78e2-4db4-8a0e-dd1fee10f9f0 None admin] RBAC: Authorizing `identity:get_group(group_id=3e9ece46115d4d7caa86f3fff59281dc)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:46.966798 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72057d-78e2-4db4-8a0e-dd1fee10f9f0 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:46.969838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 204/407] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/groups/3e9ece46115d4d7caa86f3fff59281dc => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:46.971188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72057d-78e2-4db4-8a0e-dd1fee10f9f0 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:47.060672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64068e-6ff9-4ff4-bb19-768c2fed1c6f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:47.103620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72057d-78e2-4db4-8a0e-dd1fee10f9f0 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:47.107361 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f7a35-8467-48e6-ac1d-9c75c950623b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:47.108171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f7a35-8467-48e6-ac1d-9c75c950623b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:47.108466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f7a35-8467-48e6-ac1d-9c75c950623b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:47.108749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f7a35-8467-48e6-ac1d-9c75c950623b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:47.122010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f7a35-8467-48e6-ac1d-9c75c950623b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-764f7a35-8467-48e6-ac1d-9c75c950623b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:47.134609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:47.136387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:47.137836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:47.137836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:47.137836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:47.137836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:47.137836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:47.137836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:47.137836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:47.137836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:47.137836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:47.137836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:47.137836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:47.137836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:47.140945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 204/408] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:47.142038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f7a35-8467-48e6-ac1d-9c75c950623b None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:47.179808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da9b65-d9cf-47e8-b024-a005af7ce032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:47.181560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da9b65-d9cf-47e8-b024-a005af7ce032 None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:47.182317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da9b65-d9cf-47e8-b024-a005af7ce032 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:47.182946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da9b65-d9cf-47e8-b024-a005af7ce032 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4/groups/3e9ece46115d4d7caa86f3fff59281dc/roles/a638e91b70624f669ffe76dac32323f5` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:47.186853 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da9b65-d9cf-47e8-b024-a005af7ce032 None admin] RBAC: Authorizing `identity:create_grant(project_id=8cea8011767244e0a581010033a1caf4, group_id=3e9ece46115d4d7caa86f3fff59281dc, role_id=a638e91b70624f669ffe76dac32323f5)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:47.189932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da9b65-d9cf-47e8-b024-a005af7ce032 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:47.239446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:47.291166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f7a35-8467-48e6-ac1d-9c75c950623b None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:47.329528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e71071-eaa5-4414-bdac-c11efb39f880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:47.330447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e71071-eaa5-4414-bdac-c11efb39f880 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:47.330737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e71071-eaa5-4414-bdac-c11efb39f880 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:47.331015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e71071-eaa5-4414-bdac-c11efb39f880 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:47.332241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e71071-eaa5-4414-bdac-c11efb39f880 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:47.334368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e71071-eaa5-4414-bdac-c11efb39f880 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:47.336114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 205/409] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:47 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:47.337895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e71071-eaa5-4414-bdac-c11efb39f880 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:47.343953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:47.358151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 205/410] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Jul 25 18:42:46 2022] PUT /identity/v3/projects/8cea8011767244e0a581010033a1caf4/groups/3e9ece46115d4d7caa86f3fff59281dc/roles/a638e91b70624f669ffe76dac32323f5 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:47.358151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727bbf2-b967-4e15-a0ef-ed49fd40e933 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:47.358151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727bbf2-b967-4e15-a0ef-ed49fd40e933 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:47.358151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727bbf2-b967-4e15-a0ef-ed49fd40e933 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:47.358151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 206/411] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:47.358151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727bbf2-b967-4e15-a0ef-ed49fd40e933 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:47.507833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727bbf2-b967-4e15-a0ef-ed49fd40e933 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:47.516880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e71071-eaa5-4414-bdac-c11efb39f880 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:47.575869 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84e3d5-3fb0-4510-8a5c-1477f496b7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:47.576811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84e3d5-3fb0-4510-8a5c-1477f496b7ac None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:47.577101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84e3d5-3fb0-4510-8a5c-1477f496b7ac None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:47.577388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84e3d5-3fb0-4510-8a5c-1477f496b7ac None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4/groups/3e9ece46115d4d7caa86f3fff59281dc/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:47.577758 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f86401-3226-4435-ab75-c32514b91a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:47.579190 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f86401-3226-4435-ab75-c32514b91a16 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:47.579708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f86401-3226-4435-ab75-c32514b91a16 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:47.580151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84e3d5-3fb0-4510-8a5c-1477f496b7ac None admin] RBAC: Authorizing `identity:create_grant(project_id=8cea8011767244e0a581010033a1caf4, group_id=3e9ece46115d4d7caa86f3fff59281dc, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:47.580271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f86401-3226-4435-ab75-c32514b91a16 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:47.581370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f86401-3226-4435-ab75-c32514b91a16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:47.581680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84e3d5-3fb0-4510-8a5c-1477f496b7ac None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:47.581891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f86401-3226-4435-ab75-c32514b91a16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:47.584735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f86401-3226-4435-ab75-c32514b91a16 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:47.587481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:47.600039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 206/412] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:47.601866 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313110b-543d-4727-a731-6381bbc98988 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:47.602236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313110b-543d-4727-a731-6381bbc98988 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:47.602594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313110b-543d-4727-a731-6381bbc98988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:47.629531 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-a313110b-543d-4727-a731-6381bbc98988 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:47.630834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a313110b-543d-4727-a731-6381bbc98988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:47.704649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 207/413] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Jul 25 18:42:47 2022] PUT /identity/v3/projects/8cea8011767244e0a581010033a1caf4/groups/3e9ece46115d4d7caa86f3fff59281dc/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:47.704649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84e3d5-3fb0-4510-8a5c-1477f496b7ac None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:47.742366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 207/414] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:47.841698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313110b-543d-4727-a731-6381bbc98988 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:47.860162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:47.974182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84e3d5-3fb0-4510-8a5c-1477f496b7ac None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:48.039478 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d5eba-c6be-45ca-89db-0fdbf3afe6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:48.040375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d5eba-c6be-45ca-89db-0fdbf3afe6eb None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.040665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d5eba-c6be-45ca-89db-0fdbf3afe6eb None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.040950 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d5eba-c6be-45ca-89db-0fdbf3afe6eb None admin] PATH_INFO: `/v3/groups/8aee740498a84f17bb981120ce74d892` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.042401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d5eba-c6be-45ca-89db-0fdbf3afe6eb None admin] RBAC: Authorizing `identity:get_group(group_id=8aee740498a84f17bb981120ce74d892)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:48.044009 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d5eba-c6be-45ca-89db-0fdbf3afe6eb None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:48.045848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 208/415] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:47 2022] GET /identity/v3/groups/8aee740498a84f17bb981120ce74d892 => generated 229 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:48.047854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313110b-543d-4727-a731-6381bbc98988 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:48.057989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d5eba-c6be-45ca-89db-0fdbf3afe6eb None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:48.106872 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:48.121749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d3f82-4321-40ca-9305-45fc4438f234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:48.122614 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d3f82-4321-40ca-9305-45fc4438f234 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.122901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d3f82-4321-40ca-9305-45fc4438f234 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.123183 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d3f82-4321-40ca-9305-45fc4438f234 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.166518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d3f82-4321-40ca-9305-45fc4438f234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-ca3d3f82-4321-40ca-9305-45fc4438f234 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:48.175528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:48.176688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:48.177837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:48.177837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:48.177837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:48.177837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:48.177837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:48.177837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:48.177837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:48.177837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:48.177837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:48.177837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:48.177837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:48.177837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:48.182390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 208/416] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:48.187303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d3f82-4321-40ca-9305-45fc4438f234 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:48.321017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d5eba-c6be-45ca-89db-0fdbf3afe6eb None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:48.348658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d3f82-4321-40ca-9305-45fc4438f234 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:48.355634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:48.384323 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57f90a-31a0-447e-a937-1a98b573198e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:48.384323 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57f90a-31a0-447e-a937-1a98b573198e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.384323 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57f90a-31a0-447e-a937-1a98b573198e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.384323 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57f90a-31a0-447e-a937-1a98b573198e None admin] PATH_INFO: `/v3/projects/c39adca8a83c49e4a2ab45e68b3a9724` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.384925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae57f90a-31a0-447e-a937-1a98b573198e None admin] RBAC: Authorizing `identity:get_project(project_id=c39adca8a83c49e4a2ab45e68b3a9724)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:48.384925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae57f90a-31a0-447e-a937-1a98b573198e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:48.384925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 209/417] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:48 2022] GET /identity/v3/projects/c39adca8a83c49e4a2ab45e68b3a9724 => generated 344 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:48.384925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673d3f9-bb26-4324-8445-62ba412b0f92 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.384925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673d3f9-bb26-4324-8445-62ba412b0f92 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.384925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673d3f9-bb26-4324-8445-62ba412b0f92 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.385330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 210/418] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:42:48.385330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673d3f9-bb26-4324-8445-62ba412b0f92 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:48.414384 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49379164-a90e-42b7-af70-4461532c1c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:48.416003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49379164-a90e-42b7-af70-4461532c1c32 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.416626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49379164-a90e-42b7-af70-4461532c1c32 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.417088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49379164-a90e-42b7-af70-4461532c1c32 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.417900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49379164-a90e-42b7-af70-4461532c1c32 None admin] RBAC: Adding query filter params (name=member) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:48.418313 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49379164-a90e-42b7-af70-4461532c1c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:48.421029 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49379164-a90e-42b7-af70-4461532c1c32 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:48.436916 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 209/419] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:48.439096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec1ddf-53a1-453a-bdf1-ea8bd3bfde53 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.439586 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec1ddf-53a1-453a-bdf1-ea8bd3bfde53 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.439927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec1ddf-53a1-453a-bdf1-ea8bd3bfde53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.471816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b0ec1ddf-53a1-453a-bdf1-ea8bd3bfde53 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:48.473939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b0ec1ddf-53a1-453a-bdf1-ea8bd3bfde53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:48.546091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:48.557353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673d3f9-bb26-4324-8445-62ba412b0f92 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:48.580634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 210/420] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:48.581889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec1ddf-53a1-453a-bdf1-ea8bd3bfde53 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:48.623130 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44f381-912e-4118-9ad3-5880476b1c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:48.624148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44f381-912e-4118-9ad3-5880476b1c4c None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.624440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44f381-912e-4118-9ad3-5880476b1c4c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.624711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44f381-912e-4118-9ad3-5880476b1c4c None admin] PATH_INFO: `/v3/projects/c39adca8a83c49e4a2ab45e68b3a9724/groups/8aee740498a84f17bb981120ce74d892/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.629472 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44f381-912e-4118-9ad3-5880476b1c4c None admin] RBAC: Authorizing `identity:create_grant(project_id=c39adca8a83c49e4a2ab45e68b3a9724, group_id=8aee740498a84f17bb981120ce74d892, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:48.631768 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44f381-912e-4118-9ad3-5880476b1c4c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:48.717925 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec1ddf-53a1-453a-bdf1-ea8bd3bfde53 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:48.732096 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 211/421] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Jul 25 18:42:48 2022] PUT /identity/v3/projects/c39adca8a83c49e4a2ab45e68b3a9724/groups/8aee740498a84f17bb981120ce74d892/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:42:48.734002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44f381-912e-4118-9ad3-5880476b1c4c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:48.788181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0269bd-ee4c-4bcb-9b6d-1fe518147353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:48.790119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0269bd-ee4c-4bcb-9b6d-1fe518147353 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:48.790421 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0269bd-ee4c-4bcb-9b6d-1fe518147353 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:48.790703 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0269bd-ee4c-4bcb-9b6d-1fe518147353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:48.867373 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:48.913909 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:48.962389 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0269bd-ee4c-4bcb-9b6d-1fe518147353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:48.962614 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44f381-912e-4118-9ad3-5880476b1c4c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:48.963891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0269bd-ee4c-4bcb-9b6d-1fe518147353 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:49.012450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de31b0-a830-4857-b051-30b33212dde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:49.013214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de31b0-a830-4857-b051-30b33212dde6 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.013499 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de31b0-a830-4857-b051-30b33212dde6 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.013776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de31b0-a830-4857-b051-30b33212dde6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.038238 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de31b0-a830-4857-b051-30b33212dde6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-64de31b0-a830-4857-b051-30b33212dde6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:49.055104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:49.056114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:49.056945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:49.056945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:49.056945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:49.056945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:49.056945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:49.056945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:49.056945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:49.056945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:49.056945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:49.056945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:49.056945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:49.056945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:49.056945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 212/422] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:49.064489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de31b0-a830-4857-b051-30b33212dde6 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:49.198447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 211/423] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:49.206078 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0269bd-ee4c-4bcb-9b6d-1fe518147353 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:49.283201 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de31b0-a830-4857-b051-30b33212dde6 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:49.314230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:49.357939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:49.358907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97b737-0440-4953-a1dd-a1e646bf2feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:49.359619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97b737-0440-4953-a1dd-a1e646bf2feb None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.359910 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97b737-0440-4953-a1dd-a1e646bf2feb None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.360194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97b737-0440-4953-a1dd-a1e646bf2feb None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.360969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97b737-0440-4953-a1dd-a1e646bf2feb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:49.361280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97b737-0440-4953-a1dd-a1e646bf2feb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:49.362771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97b737-0440-4953-a1dd-a1e646bf2feb None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:49.385500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 213/424] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:49.386588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97b737-0440-4953-a1dd-a1e646bf2feb None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:49.440547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0269bd-ee4c-4bcb-9b6d-1fe518147353 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:49.520450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97b737-0440-4953-a1dd-a1e646bf2feb None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:49.551170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345b1d1-f482-4c45-9a5d-c12c74b7c689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:49.552184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345b1d1-f482-4c45-9a5d-c12c74b7c689 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.552583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345b1d1-f482-4c45-9a5d-c12c74b7c689 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.553058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345b1d1-f482-4c45-9a5d-c12c74b7c689 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.559372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345b1d1-f482-4c45-9a5d-c12c74b7c689 None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:49.560994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345b1d1-f482-4c45-9a5d-c12c74b7c689 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:49.562197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb10702-1b61-4b40-b1fc-ca19962723dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:49.567415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 212/425] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:49 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:49.567686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb10702-1b61-4b40-b1fc-ca19962723dc None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.568812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb10702-1b61-4b40-b1fc-ca19962723dc None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.569604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb10702-1b61-4b40-b1fc-ca19962723dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.575494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345b1d1-f482-4c45-9a5d-c12c74b7c689 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:49.644457 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:49.661480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb10702-1b61-4b40-b1fc-ca19962723dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:49.664423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb10702-1b61-4b40-b1fc-ca19962723dc None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:49.790103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345b1d1-f482-4c45-9a5d-c12c74b7c689 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:49.883833 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0937a1-fba2-4eeb-8964-eaf1c3e304ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:49.884615 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0937a1-fba2-4eeb-8964-eaf1c3e304ad None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.884906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0937a1-fba2-4eeb-8964-eaf1c3e304ad None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.885180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0937a1-fba2-4eeb-8964-eaf1c3e304ad None admin] PATH_INFO: `/v3/groups/3e9ece46115d4d7caa86f3fff59281dc` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.888503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0937a1-fba2-4eeb-8964-eaf1c3e304ad None admin] RBAC: Authorizing `identity:get_group(group_id=3e9ece46115d4d7caa86f3fff59281dc)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:49.889970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0937a1-fba2-4eeb-8964-eaf1c3e304ad None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:49.894943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 213/426] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:49 2022] GET /identity/v3/groups/3e9ece46115d4d7caa86f3fff59281dc => generated 226 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:49.901213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0937a1-fba2-4eeb-8964-eaf1c3e304ad None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:49.921261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 214/427] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:49.927324 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c3978-e4bc-48c3-98bd-06bca8dfaf7b None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:49.928528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c3978-e4bc-48c3-98bd-06bca8dfaf7b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:49.928822 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c3978-e4bc-48c3-98bd-06bca8dfaf7b None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:49.931137 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 215/428] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:49.931137 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c3978-e4bc-48c3-98bd-06bca8dfaf7b None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:49.946520 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:50.063865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0937a1-fba2-4eeb-8964-eaf1c3e304ad None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:50.064730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:50.109355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c3978-e4bc-48c3-98bd-06bca8dfaf7b None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:50.113291 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb9362-261c-454c-a610-9e0d248045b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:50.114181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb9362-261c-454c-a610-9e0d248045b4 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.114658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb9362-261c-454c-a610-9e0d248045b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.115853 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb9362-261c-454c-a610-9e0d248045b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.117320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb9362-261c-454c-a610-9e0d248045b4 None admin] RBAC: Adding query filter params (group.id=3e9ece46115d4d7caa86f3fff59281dc, role.id=9dc988eb162e47e48df810053eecb165, scope.project.id=f0435acd015c454f926f9318bb5503b2) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:50.117822 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb9362-261c-454c-a610-9e0d248045b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:50.119932 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb9362-261c-454c-a610-9e0d248045b4 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:50.128837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 214/429] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 18:42:49 2022] GET /identity/v3/role_assignments?group.id=3e9ece46115d4d7caa86f3fff59281dc&scope.project.id=f0435acd015c454f926f9318bb5503b2&role.id=9dc988eb162e47e48df810053eecb165 => generated 611 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:50.133007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7902e-107c-40b9-9a48-35dc8618124f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.133770 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7902e-107c-40b9-9a48-35dc8618124f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.134581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7902e-107c-40b9-9a48-35dc8618124f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.157316 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2c380-4a99-4f6e-8473-867b687f4730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:50.157316 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2c380-4a99-4f6e-8473-867b687f4730 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.157316 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2c380-4a99-4f6e-8473-867b687f4730 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.157316 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2c380-4a99-4f6e-8473-867b687f4730 None admin] PATH_INFO: `/v3/projects/f0435acd015c454f926f9318bb5503b2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.158557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2c380-4a99-4f6e-8473-867b687f4730 None admin] RBAC: Authorizing `identity:get_project(project_id=f0435acd015c454f926f9318bb5503b2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:50.160717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2c380-4a99-4f6e-8473-867b687f4730 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:50.162830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 216/430] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:49 2022] GET /identity/v3/projects/f0435acd015c454f926f9318bb5503b2 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:50.171641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2c380-4a99-4f6e-8473-867b687f4730 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:50.180712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-bba7902e-107c-40b9-9a48-35dc8618124f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:50.183495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bba7902e-107c-40b9-9a48-35dc8618124f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:50.289477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 215/431] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:50.344039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7902e-107c-40b9-9a48-35dc8618124f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:50.359045 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2c380-4a99-4f6e-8473-867b687f4730 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:50.368816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:50.398588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ba2f9-0981-480e-8aa8-4b49bd6f6ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:50.399462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ba2f9-0981-480e-8aa8-4b49bd6f6ecd None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.399750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ba2f9-0981-480e-8aa8-4b49bd6f6ecd None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.400034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ba2f9-0981-480e-8aa8-4b49bd6f6ecd None admin] PATH_INFO: `/v3/groups/3e9ece46115d4d7caa86f3fff59281dc` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.401321 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ba2f9-0981-480e-8aa8-4b49bd6f6ecd None admin] RBAC: Authorizing `identity:get_group(group_id=3e9ece46115d4d7caa86f3fff59281dc)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:50.402863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ba2f9-0981-480e-8aa8-4b49bd6f6ecd None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:50.404557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 217/432] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:50 2022] GET /identity/v3/groups/3e9ece46115d4d7caa86f3fff59281dc => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:50.411306 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ba2f9-0981-480e-8aa8-4b49bd6f6ecd None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:50.432847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:50.515587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7902e-107c-40b9-9a48-35dc8618124f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:50.585342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4215724-94f6-4ae4-94a7-225dfd865ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:50.585574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ba2f9-0981-480e-8aa8-4b49bd6f6ecd None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:50.586186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4215724-94f6-4ae4-94a7-225dfd865ebc None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.587680 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4215724-94f6-4ae4-94a7-225dfd865ebc None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.587996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4215724-94f6-4ae4-94a7-225dfd865ebc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.602849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4215724-94f6-4ae4-94a7-225dfd865ebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-b4215724-94f6-4ae4-94a7-225dfd865ebc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:50.628956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:50.629955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:50.631176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:50.631176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:50.631176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:50.631176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:50.631176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:50.631176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:50.631176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:50.631176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:50.631176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:50.631176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:50.631176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:50.631176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:50.631176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 216/433] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:50.632058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32976f18-d337-46c6-8231-1447c459b336 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.632653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32976f18-d337-46c6-8231-1447c459b336 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.632653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32976f18-d337-46c6-8231-1447c459b336 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.633829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 217/434] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:50.639893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5cc8-c438-4062-9887-96edd32c9507 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.640309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5cc8-c438-4062-9887-96edd32c9507 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.640481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5cc8-c438-4062-9887-96edd32c9507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.652609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383d4e2-7506-468b-8de0-1310f3a2c980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:50.654839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383d4e2-7506-468b-8de0-1310f3a2c980 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.655342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383d4e2-7506-468b-8de0-1310f3a2c980 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.655843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383d4e2-7506-468b-8de0-1310f3a2c980 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.656723 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383d4e2-7506-468b-8de0-1310f3a2c980 None admin] RBAC: Adding query filter params (group.id=3e9ece46115d4d7caa86f3fff59281dc, role.id=a638e91b70624f669ffe76dac32323f5, scope.project.id=f0435acd015c454f926f9318bb5503b2) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:50.657070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383d4e2-7506-468b-8de0-1310f3a2c980 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:50.660204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383d4e2-7506-468b-8de0-1310f3a2c980 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:50.670002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 218/435] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 18:42:50 2022] GET /identity/v3/role_assignments?group.id=3e9ece46115d4d7caa86f3fff59281dc&scope.project.id=f0435acd015c454f926f9318bb5503b2&role.id=a638e91b70624f669ffe76dac32323f5 => generated 611 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:50.670002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383d4e2-7506-468b-8de0-1310f3a2c980 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:50.680343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-521d5cc8-c438-4062-9887-96edd32c9507 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:50.682285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-521d5cc8-c438-4062-9887-96edd32c9507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:50.776761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:50.778709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:50.784088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 218/436] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:50.809249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383d4e2-7506-468b-8de0-1310f3a2c980 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:50.817332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d5cc8-c438-4062-9887-96edd32c9507 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:50.867498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b18f4-b793-4224-8983-a0c5eb5bccc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:50.869036 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b18f4-b793-4224-8983-a0c5eb5bccc7 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:50.869786 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b18f4-b793-4224-8983-a0c5eb5bccc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:50.870254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b18f4-b793-4224-8983-a0c5eb5bccc7 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:50.871499 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b18f4-b793-4224-8983-a0c5eb5bccc7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:50.872586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b18f4-b793-4224-8983-a0c5eb5bccc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:50.874527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b18f4-b793-4224-8983-a0c5eb5bccc7 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:50.888574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 219/437] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:50.899435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b18f4-b793-4224-8983-a0c5eb5bccc7 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:50.994361 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d5cc8-c438-4062-9887-96edd32c9507 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:51.051082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12269d91-d6c0-4df4-9e25-dfec8c8829d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:51.051985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12269d91-d6c0-4df4-9e25-dfec8c8829d7 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:51.052200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12269d91-d6c0-4df4-9e25-dfec8c8829d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:51.052488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12269d91-d6c0-4df4-9e25-dfec8c8829d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:51.062310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b18f4-b793-4224-8983-a0c5eb5bccc7 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:51.069999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12269d91-d6c0-4df4-9e25-dfec8c8829d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-12269d91-d6c0-4df4-9e25-dfec8c8829d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:51.079702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:51.080804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:51.081638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:51.081638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:51.081638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:51.081638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:51.081638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:51.081638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:51.081638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:51.081638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:51.081638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:51.081638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:51.081638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 18:42:51.081638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:51.081638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 219/438] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:51.088580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12269d91-d6c0-4df4-9e25-dfec8c8829d7 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:51.116563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed5bf2-3f5d-49f0-9363-990ab75383e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:51.117881 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed5bf2-3f5d-49f0-9363-990ab75383e8 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:51.118602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed5bf2-3f5d-49f0-9363-990ab75383e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:51.119362 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed5bf2-3f5d-49f0-9363-990ab75383e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:51.132408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:51.206205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:51.235285 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed5bf2-3f5d-49f0-9363-990ab75383e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:51.238126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed5bf2-3f5d-49f0-9363-990ab75383e8 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:51.272204 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12269d91-d6c0-4df4-9e25-dfec8c8829d7 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:51.322747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372ed22-4680-4140-a3da-b1d1abca1f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:51.323572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372ed22-4680-4140-a3da-b1d1abca1f91 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:51.323866 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372ed22-4680-4140-a3da-b1d1abca1f91 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:51.324144 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372ed22-4680-4140-a3da-b1d1abca1f91 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:51.324899 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372ed22-4680-4140-a3da-b1d1abca1f91 None admin] RBAC: Adding query filter params (name=member) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:51.325209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372ed22-4680-4140-a3da-b1d1abca1f91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:51.326616 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372ed22-4680-4140-a3da-b1d1abca1f91 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:51.336209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 220/439] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:42:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:42:51.338045 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b1e5b-dd69-4d14-85a8-256e77509b95 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:51.338358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b1e5b-dd69-4d14-85a8-256e77509b95 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:51.338719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b1e5b-dd69-4d14-85a8-256e77509b95 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:51.340305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 221/440] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:51.343271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b1e5b-dd69-4d14-85a8-256e77509b95 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:51.382126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 220/441] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:51.383457 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd35c3f-da22-47cf-b9f0-b4852458f2a8 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:51.384166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd35c3f-da22-47cf-b9f0-b4852458f2a8 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:51.384166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd35c3f-da22-47cf-b9f0-b4852458f2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:51.403094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-1fd35c3f-da22-47cf-b9f0-b4852458f2a8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:51.404382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1fd35c3f-da22-47cf-b9f0-b4852458f2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:51.440266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:51.473506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b1e5b-dd69-4d14-85a8-256e77509b95 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:51.483347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 221/442] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:51.484656 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd35c3f-da22-47cf-b9f0-b4852458f2a8 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:51.508736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:51.527363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bd356-de1c-4914-b43e-7e2c1cd7c4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:51.529518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bd356-de1c-4914-b43e-7e2c1cd7c4ec None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:51.529806 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bd356-de1c-4914-b43e-7e2c1cd7c4ec None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:51.530091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bd356-de1c-4914-b43e-7e2c1cd7c4ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:51.601793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd35c3f-da22-47cf-b9f0-b4852458f2a8 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:51.612177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bd356-de1c-4914-b43e-7e2c1cd7c4ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:51.613650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bd356-de1c-4914-b43e-7e2c1cd7c4ec None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:51.647610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3c9f5-ea93-453d-9b7c-a06f78543252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:51.648494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3c9f5-ea93-453d-9b7c-a06f78543252 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:51.648782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3c9f5-ea93-453d-9b7c-a06f78543252 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:51.649068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3c9f5-ea93-453d-9b7c-a06f78543252 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:51.650313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3c9f5-ea93-453d-9b7c-a06f78543252 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:51.651795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3c9f5-ea93-453d-9b7c-a06f78543252 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:51.653521 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 222/443] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:51 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:51.654395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3c9f5-ea93-453d-9b7c-a06f78543252 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:51.694711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:51.733104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 222/444] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:51.733980 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bd356-de1c-4914-b43e-7e2c1cd7c4ec None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:51.756798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:51.780855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3c9f5-ea93-453d-9b7c-a06f78543252 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:51.825195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180775cc-7501-423e-bf01-6520564c0d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:51.826003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180775cc-7501-423e-bf01-6520564c0d83 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:51.826287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180775cc-7501-423e-bf01-6520564c0d83 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:51.826598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180775cc-7501-423e-bf01-6520564c0d83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:51.837759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180775cc-7501-423e-bf01-6520564c0d83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-180775cc-7501-423e-bf01-6520564c0d83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:51.845505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:51.846539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:51.847781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:51.847781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:51.847781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:51.847781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:51.847781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:51.847781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:51.847781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:51.847781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:51.847781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:51.847781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:51.847781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:42:51.847781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:51.847781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 223/445] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:42:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:51.849022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180775cc-7501-423e-bf01-6520564c0d83 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:51.866840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bd356-de1c-4914-b43e-7e2c1cd7c4ec None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:51.909501 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f11b3a-f275-4773-9778-17e812b0c154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:51.910509 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f11b3a-f275-4773-9778-17e812b0c154 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:51.910792 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f11b3a-f275-4773-9778-17e812b0c154 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:51.911079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f11b3a-f275-4773-9778-17e812b0c154 None admin] PATH_INFO: `/v3/groups/3e9ece46115d4d7caa86f3fff59281dc` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:51.912772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f11b3a-f275-4773-9778-17e812b0c154 None admin] RBAC: Authorizing `identity:get_group(group_id=3e9ece46115d4d7caa86f3fff59281dc)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:51.914742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f11b3a-f275-4773-9778-17e812b0c154 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:51.916818 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 223/446] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:51 2022] GET /identity/v3/groups/3e9ece46115d4d7caa86f3fff59281dc => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:51.917431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f11b3a-f275-4773-9778-17e812b0c154 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:51.961993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:51.968145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180775cc-7501-423e-bf01-6520564c0d83 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:52.004863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72003c2-8ccb-4b07-877e-93a05ae775d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:52.006303 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72003c2-8ccb-4b07-877e-93a05ae775d4 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.006737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72003c2-8ccb-4b07-877e-93a05ae775d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.007109 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72003c2-8ccb-4b07-877e-93a05ae775d4 None admin] PATH_INFO: `/v3/projects/8cea8011767244e0a581010033a1caf4` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.008188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72003c2-8ccb-4b07-877e-93a05ae775d4 None admin] RBAC: Authorizing `identity:get_project(project_id=8cea8011767244e0a581010033a1caf4)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:52.009681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72003c2-8ccb-4b07-877e-93a05ae775d4 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:52.011503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 224/447] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:51 2022] GET /identity/v3/projects/8cea8011767244e0a581010033a1caf4 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:52.012296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72003c2-8ccb-4b07-877e-93a05ae775d4 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:52.015158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:52.035025 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f11b3a-f275-4773-9778-17e812b0c154 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:52.083495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9f559-6da7-476b-a3f3-31246ed58f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:52.084279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9f559-6da7-476b-a3f3-31246ed58f5f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.084762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9f559-6da7-476b-a3f3-31246ed58f5f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.085215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9f559-6da7-476b-a3f3-31246ed58f5f None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.085984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9f559-6da7-476b-a3f3-31246ed58f5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:52.086292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9f559-6da7-476b-a3f3-31246ed58f5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:52.087826 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9f559-6da7-476b-a3f3-31246ed58f5f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:52.097082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 224/448] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:42:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:52.097976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9f559-6da7-476b-a3f3-31246ed58f5f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:52.134239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72003c2-8ccb-4b07-877e-93a05ae775d4 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:52.176432 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea446e4-c272-4710-abc1-b295d6d33af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:52.177117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea446e4-c272-4710-abc1-b295d6d33af9 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.177408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea446e4-c272-4710-abc1-b295d6d33af9 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.178429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea446e4-c272-4710-abc1-b295d6d33af9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.179312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea446e4-c272-4710-abc1-b295d6d33af9 None admin] RBAC: Adding query filter params (group.id=3e9ece46115d4d7caa86f3fff59281dc, role.id=a638e91b70624f669ffe76dac32323f5, scope.project.id=8cea8011767244e0a581010033a1caf4) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:52.179850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea446e4-c272-4710-abc1-b295d6d33af9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:52.181117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea446e4-c272-4710-abc1-b295d6d33af9 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:52.187815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 225/449] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 18:42:52 2022] GET /identity/v3/role_assignments?group.id=3e9ece46115d4d7caa86f3fff59281dc&scope.project.id=8cea8011767244e0a581010033a1caf4&role.id=a638e91b70624f669ffe76dac32323f5 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:52.188672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea446e4-c272-4710-abc1-b295d6d33af9 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:52.233618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:52.238927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9f559-6da7-476b-a3f3-31246ed58f5f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:52.292126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a6476-d736-4a7a-b758-cf389dad4059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:52.292957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a6476-d736-4a7a-b758-cf389dad4059 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.293250 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a6476-d736-4a7a-b758-cf389dad4059 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.293532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a6476-d736-4a7a-b758-cf389dad4059 None admin] PATH_INFO: `/v3/groups/3e9ece46115d4d7caa86f3fff59281dc` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.294926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a6476-d736-4a7a-b758-cf389dad4059 None admin] RBAC: Authorizing `identity:get_group(group_id=3e9ece46115d4d7caa86f3fff59281dc)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:52.296563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a6476-d736-4a7a-b758-cf389dad4059 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:52.304084 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 225/450] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:52 2022] GET /identity/v3/groups/3e9ece46115d4d7caa86f3fff59281dc => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:52.305810 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a6476-d736-4a7a-b758-cf389dad4059 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:52.331167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:52.360773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea446e4-c272-4710-abc1-b295d6d33af9 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:52.411339 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f260e-87b1-4509-aaf7-20e375f0ebc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:52.412310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f260e-87b1-4509-aaf7-20e375f0ebc5 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.412706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f260e-87b1-4509-aaf7-20e375f0ebc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.413000 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f260e-87b1-4509-aaf7-20e375f0ebc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.449678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a6476-d736-4a7a-b758-cf389dad4059 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:52.493086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562adb32-6780-4ed4-8d33-9a24e36bb792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:52.494128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562adb32-6780-4ed4-8d33-9a24e36bb792 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.494642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562adb32-6780-4ed4-8d33-9a24e36bb792 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.495105 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562adb32-6780-4ed4-8d33-9a24e36bb792 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.496845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562adb32-6780-4ed4-8d33-9a24e36bb792 None admin] RBAC: Adding query filter params (group.id=3e9ece46115d4d7caa86f3fff59281dc, role.id=9dc988eb162e47e48df810053eecb165, scope.project.id=8cea8011767244e0a581010033a1caf4) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:52.497417 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562adb32-6780-4ed4-8d33-9a24e36bb792 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:52.499739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562adb32-6780-4ed4-8d33-9a24e36bb792 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:52.504802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f260e-87b1-4509-aaf7-20e375f0ebc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:52.506713 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f260e-87b1-4509-aaf7-20e375f0ebc5 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:52.507139 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 226/451] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 18:42:52 2022] GET /identity/v3/role_assignments?group.id=3e9ece46115d4d7caa86f3fff59281dc&scope.project.id=8cea8011767244e0a581010033a1caf4&role.id=9dc988eb162e47e48df810053eecb165 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:52.539366 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:52.656437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 226/452] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:42:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:42:52.662957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562adb32-6780-4ed4-8d33-9a24e36bb792 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:52.768051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:52.788416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562adb32-6780-4ed4-8d33-9a24e36bb792 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:52.832364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9eaf32-719b-4397-9bab-d2267d12e926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:52.833718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9eaf32-719b-4397-9bab-d2267d12e926 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:52.834201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9eaf32-719b-4397-9bab-d2267d12e926 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:52.834660 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9eaf32-719b-4397-9bab-d2267d12e926 None admin] PATH_INFO: `/v3/projects/c39adca8a83c49e4a2ab45e68b3a9724` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:52.836546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9eaf32-719b-4397-9bab-d2267d12e926 None admin] RBAC: Authorizing `identity:get_project(project_id=c39adca8a83c49e4a2ab45e68b3a9724)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:52.838396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9eaf32-719b-4397-9bab-d2267d12e926 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:52.841556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 227/453] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:42:52 2022] GET /identity/v3/projects/c39adca8a83c49e4a2ab45e68b3a9724 => generated 344 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:52.847762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f260e-87b1-4509-aaf7-20e375f0ebc5 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:52.976347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f260e-87b1-4509-aaf7-20e375f0ebc5 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:53.008424 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:53.016859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb011c-2ef6-4cbd-b58a-17ee97d153fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:53.017743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb011c-2ef6-4cbd-b58a-17ee97d153fd None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:53.018032 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb011c-2ef6-4cbd-b58a-17ee97d153fd None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:53.018313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb011c-2ef6-4cbd-b58a-17ee97d153fd None admin] PATH_INFO: `/v3/groups/8aee740498a84f17bb981120ce74d892` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:53.019625 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb011c-2ef6-4cbd-b58a-17ee97d153fd None admin] RBAC: Authorizing `identity:get_group(group_id=8aee740498a84f17bb981120ce74d892)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:53.024192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb011c-2ef6-4cbd-b58a-17ee97d153fd None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:53.024192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 227/454] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 18:42:52 2022] GET /identity/v3/groups/8aee740498a84f17bb981120ce74d892 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:53.030394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9eaf32-719b-4397-9bab-d2267d12e926 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:53.170381 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9eaf32-719b-4397-9bab-d2267d12e926 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:53.213507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda33fd2-ce46-4d6c-8cc8-150a0618d224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:53.214174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda33fd2-ce46-4d6c-8cc8-150a0618d224 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:53.214489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda33fd2-ce46-4d6c-8cc8-150a0618d224 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:53.214772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda33fd2-ce46-4d6c-8cc8-150a0618d224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:53.215605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda33fd2-ce46-4d6c-8cc8-150a0618d224 None admin] RBAC: Adding query filter params (group.id=8aee740498a84f17bb981120ce74d892, role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=c39adca8a83c49e4a2ab45e68b3a9724) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:53.215913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda33fd2-ce46-4d6c-8cc8-150a0618d224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:53.217357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda33fd2-ce46-4d6c-8cc8-150a0618d224 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:53.224085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 228/455] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 18:42:53 2022] GET /identity/v3/role_assignments?group.id=8aee740498a84f17bb981120ce74d892&scope.project.id=c39adca8a83c49e4a2ab45e68b3a9724&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:55.776512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9a435-c2de-4247-8ad6-4443019386c9 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:55.777547 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9a435-c2de-4247-8ad6-4443019386c9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:55.778040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9a435-c2de-4247-8ad6-4443019386c9 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:55.779991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 228/456] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:55.786140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44389e60-cd8d-497f-8e8c-f2eefd0010fa None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:55.787834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44389e60-cd8d-497f-8e8c-f2eefd0010fa None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:55.788398 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44389e60-cd8d-497f-8e8c-f2eefd0010fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:55.815771 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-44389e60-cd8d-497f-8e8c-f2eefd0010fa None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:55.817356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-44389e60-cd8d-497f-8e8c-f2eefd0010fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:55.835577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:55.932910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 229/457] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:55.970631 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9a435-c2de-4247-8ad6-4443019386c9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:56.144966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9a435-c2de-4247-8ad6-4443019386c9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:56.207179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced332b-d639-4baa-8faa-216f7d6191d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:56.209399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced332b-d639-4baa-8faa-216f7d6191d5 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.209696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced332b-d639-4baa-8faa-216f7d6191d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.209976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced332b-d639-4baa-8faa-216f7d6191d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.224059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced332b-d639-4baa-8faa-216f7d6191d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:56.226600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced332b-d639-4baa-8faa-216f7d6191d5 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-3ced332b-d639-4baa-8faa-216f7d6191d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:42:56.251937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:56.253735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:56.258034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:42:56.258034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:42:56.258034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:42:56.258034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:56.258034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 229/458] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:56.262569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44389e60-cd8d-497f-8e8c-f2eefd0010fa None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:56.437515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44389e60-cd8d-497f-8e8c-f2eefd0010fa None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:56.492754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd057fd-b886-42e6-b6b0-78531fc97662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:56.493462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd057fd-b886-42e6-b6b0-78531fc97662 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.493756 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd057fd-b886-42e6-b6b0-78531fc97662 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.494042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd057fd-b886-42e6-b6b0-78531fc97662 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.494819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd057fd-b886-42e6-b6b0-78531fc97662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:56.495135 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd057fd-b886-42e6-b6b0-78531fc97662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:56.496526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd057fd-b886-42e6-b6b0-78531fc97662 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:56.499585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cd5d0-9334-4222-9107-f8daa33666a4 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.500075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cd5d0-9334-4222-9107-f8daa33666a4 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.500504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cd5d0-9334-4222-9107-f8daa33666a4 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.506834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 230/459] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:56.509084 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402eb51-8412-4f9b-b22e-8c9a1886ddcf None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.509530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402eb51-8412-4f9b-b22e-8c9a1886ddcf None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.509977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402eb51-8412-4f9b-b22e-8c9a1886ddcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.518907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 230/460] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:56.527603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd057fd-b886-42e6-b6b0-78531fc97662 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:56.549586 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:56.549779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-c402eb51-8412-4f9b-b22e-8c9a1886ddcf None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:56.550992 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c402eb51-8412-4f9b-b22e-8c9a1886ddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:56.612527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:56.639242 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 231/461] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:56.640687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e9790-81fe-440f-8937-c7a31872fb49 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.640994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e9790-81fe-440f-8937-c7a31872fb49 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.641267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e9790-81fe-440f-8937-c7a31872fb49 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.643408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 232/462] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:56.659841 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bfd94-1618-483d-bc3d-23a37511daa1 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.660198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bfd94-1618-483d-bc3d-23a37511daa1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.660493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bfd94-1618-483d-bc3d-23a37511daa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.691107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-686bfd94-1618-483d-bc3d-23a37511daa1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:56.692823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-686bfd94-1618-483d-bc3d-23a37511daa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:56.704116 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd057fd-b886-42e6-b6b0-78531fc97662 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:56.768687 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9b7ec-ec23-4c26-b496-f0f95e6a179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:56.770076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9b7ec-ec23-4c26-b496-f0f95e6a179c None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.770739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9b7ec-ec23-4c26-b496-f0f95e6a179c None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.772549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9b7ec-ec23-4c26-b496-f0f95e6a179c None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.774406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9b7ec-ec23-4c26-b496-f0f95e6a179c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:56.776565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9b7ec-ec23-4c26-b496-f0f95e6a179c None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:56.782975 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 233/463] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:42:56.795792 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bfd94-1618-483d-bc3d-23a37511daa1 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:56.922112 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bfd94-1618-483d-bc3d-23a37511daa1 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:56.933232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:56.960442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab8086-3881-4ae6-956c-77fd5d83b67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:56.961213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab8086-3881-4ae6-956c-77fd5d83b67a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:56.961510 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab8086-3881-4ae6-956c-77fd5d83b67a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:56.961790 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab8086-3881-4ae6-956c-77fd5d83b67a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:56.975554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab8086-3881-4ae6-956c-77fd5d83b67a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:56.977103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab8086-3881-4ae6-956c-77fd5d83b67a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-f2ab8086-3881-4ae6-956c-77fd5d83b67a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:42:57.001789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:57.143014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:57.144465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:42:57.144465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:42:57.144465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:42:57.144465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:57.144465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 234/464] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:57.144465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab8086-3881-4ae6-956c-77fd5d83b67a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:57.144465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab8086-3881-4ae6-956c-77fd5d83b67a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:57.168174 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8fccf-eb70-466e-860e-691d4b06771e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:57.168988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8fccf-eb70-466e-860e-691d4b06771e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.169349 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8fccf-eb70-466e-860e-691d4b06771e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.169739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8fccf-eb70-466e-860e-691d4b06771e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.186068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8fccf-eb70-466e-860e-691d4b06771e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:57.189406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8fccf-eb70-466e-860e-691d4b06771e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-78a8fccf-eb70-466e-860e-691d4b06771e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:42:57.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:57.226040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:57.233640 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:42:57.233640 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:42:57.233640 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:42:57.233640 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:57.233640 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 235/465] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:57.233640 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8fccf-eb70-466e-860e-691d4b06771e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:57.352233 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8fccf-eb70-466e-860e-691d4b06771e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:57.387616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03add3ac-245f-49ff-b752-d75af05e54d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:57.388305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03add3ac-245f-49ff-b752-d75af05e54d2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.388591 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03add3ac-245f-49ff-b752-d75af05e54d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.388873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03add3ac-245f-49ff-b752-d75af05e54d2 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.389621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03add3ac-245f-49ff-b752-d75af05e54d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:57.389933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03add3ac-245f-49ff-b752-d75af05e54d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:57.391444 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03add3ac-245f-49ff-b752-d75af05e54d2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:57.406481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 236/466] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:57.413108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03add3ac-245f-49ff-b752-d75af05e54d2 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:57.448294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 231/467] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:56 2022] POST /identity/v3/users => generated 246 bytes in 922 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:42:57.449809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778629f-f2a2-403b-92d3-a1cc7c0f08d8 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.450103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778629f-f2a2-403b-92d3-a1cc7c0f08d8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.450404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778629f-f2a2-403b-92d3-a1cc7c0f08d8 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.457344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 232/468] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:57.457344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778629f-f2a2-403b-92d3-a1cc7c0f08d8 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:57.554832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:57.601144 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:57.608141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:57.636922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03add3ac-245f-49ff-b752-d75af05e54d2 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:57.675898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778629f-f2a2-403b-92d3-a1cc7c0f08d8 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:57.685784 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dd2b1-b07c-4675-b584-0a7a5bf587ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:57.686940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dd2b1-b07c-4675-b584-0a7a5bf587ee None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.687797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dd2b1-b07c-4675-b584-0a7a5bf587ee None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.687945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dd2b1-b07c-4675-b584-0a7a5bf587ee None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.689326 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dd2b1-b07c-4675-b584-0a7a5bf587ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:57.689903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dd2b1-b07c-4675-b584-0a7a5bf587ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:57.694077 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dd2b1-b07c-4675-b584-0a7a5bf587ee None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:57.710314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 237/469] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:57.711262 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8788df9-a294-4639-85d5-ba1596a8c2c8 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.711564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8788df9-a294-4639-85d5-ba1596a8c2c8 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.711846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8788df9-a294-4639-85d5-ba1596a8c2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.727239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eab1a7-4ea9-4f86-8332-c9c914ab6b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:57.727952 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eab1a7-4ea9-4f86-8332-c9c914ab6b5c None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.728241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eab1a7-4ea9-4f86-8332-c9c914ab6b5c None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.729124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eab1a7-4ea9-4f86-8332-c9c914ab6b5c None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.729857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eab1a7-4ea9-4f86-8332-c9c914ab6b5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:57.731749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eab1a7-4ea9-4f86-8332-c9c914ab6b5c None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:57.734977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-b8788df9-a294-4639-85d5-ba1596a8c2c8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:57.736283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b8788df9-a294-4639-85d5-ba1596a8c2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:57.811683 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 238/470] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:57.816205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8d4b6-9d64-4fc2-8888-5a1c4d4e0b18 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.816508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8d4b6-9d64-4fc2-8888-5a1c4d4e0b18 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.816833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8d4b6-9d64-4fc2-8888-5a1c4d4e0b18 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.819014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 239/471] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:57.819438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8d4b6-9d64-4fc2-8888-5a1c4d4e0b18 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:57.889523 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:57.917653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 233/472] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:57 2022] POST /identity/v3/users => generated 248 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:57.919372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531a10a-961c-4d20-a382-1ef20a67d87e None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.919890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531a10a-961c-4d20-a382-1ef20a67d87e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.920404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531a10a-961c-4d20-a382-1ef20a67d87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.939237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8d4b6-9d64-4fc2-8888-5a1c4d4e0b18 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:57.955037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3531a10a-961c-4d20-a382-1ef20a67d87e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:57.957668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3531a10a-961c-4d20-a382-1ef20a67d87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:57.984331 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3091f23-1d61-4167-a720-1b2f786bb93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:57.985500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3091f23-1d61-4167-a720-1b2f786bb93c None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:57.986027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3091f23-1d61-4167-a720-1b2f786bb93c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:57.986576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3091f23-1d61-4167-a720-1b2f786bb93c None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:57.987577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3091f23-1d61-4167-a720-1b2f786bb93c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:57.989477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3091f23-1d61-4167-a720-1b2f786bb93c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:58.061363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 234/473] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:42:58.061363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531a10a-961c-4d20-a382-1ef20a67d87e None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:58.124021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:58.138978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 240/474] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:57 2022] POST /identity/v3/users => generated 248 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:58.139996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3091f23-1d61-4167-a720-1b2f786bb93c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:58.219790 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531a10a-961c-4d20-a382-1ef20a67d87e None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:58.266088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708682df-3239-47e7-9fbe-5dd731e74c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:58.267414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708682df-3239-47e7-9fbe-5dd731e74c13 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:58.267714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708682df-3239-47e7-9fbe-5dd731e74c13 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:58.267999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708682df-3239-47e7-9fbe-5dd731e74c13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:58.282466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708682df-3239-47e7-9fbe-5dd731e74c13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:58.284161 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708682df-3239-47e7-9fbe-5dd731e74c13 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-708682df-3239-47e7-9fbe-5dd731e74c13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:42:58.310394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:58.311318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:58.312138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:42:58.312138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:42:58.312138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:42:58.312138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:42:58.312138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 235/475] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:58.314794 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708682df-3239-47e7-9fbe-5dd731e74c13 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:58.326857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3091f23-1d61-4167-a720-1b2f786bb93c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:58.348782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:58.370478 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c5c9f-dbcf-4cc7-935f-1ef0168e91f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:58.372734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c5c9f-dbcf-4cc7-935f-1ef0168e91f0 None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:58.372734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c5c9f-dbcf-4cc7-935f-1ef0168e91f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:58.372734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c5c9f-dbcf-4cc7-935f-1ef0168e91f0 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:58.373053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c5c9f-dbcf-4cc7-935f-1ef0168e91f0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:58.375173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c5c9f-dbcf-4cc7-935f-1ef0168e91f0 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:58.418034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.sql.core [None req-314c5c9f-dbcf-4cc7-935f-1ef0168e91f0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 18:42:58.418034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 18:42:58.418034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [parameters: {'id': '067081736a6d434987983625857da6c8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 18:42:58.418034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86905) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-314c5c9f-dbcf-4cc7-935f-1ef0168e91f0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application conn.query(q) Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application result.read() Jul 25 18:42:58.438350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return ref.to_dict() Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 18:42:58.439680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application next(self.gen) Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application yield resource Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application next(self.gen) Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise self.value Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application session.commit() Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 18:42:58.440679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self.session.flush() Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application compat.raise_( Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 18:42:58.441582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 18:42:58.442528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application conn.query(q) Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application result.read() Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 18:42:58.444781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [parameters: {'id': '067081736a6d434987983625857da6c8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:58.452657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 18:42:58.454971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:58.457051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 241/476] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Jul 25 18:42:58 2022] POST /identity/v3/roles => generated 144 bytes in 300 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:58.457051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c5c9f-dbcf-4cc7-935f-1ef0168e91f0 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:58.488539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:58.494488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708682df-3239-47e7-9fbe-5dd731e74c13 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:58.546494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9b862-3f53-4d5d-9c16-700e7a09739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:58.547273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9b862-3f53-4d5d-9c16-700e7a09739d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:58.547582 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9b862-3f53-4d5d-9c16-700e7a09739d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:58.547864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9b862-3f53-4d5d-9c16-700e7a09739d None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:58.548603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9b862-3f53-4d5d-9c16-700e7a09739d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:58.548915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9b862-3f53-4d5d-9c16-700e7a09739d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:58.550300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9b862-3f53-4d5d-9c16-700e7a09739d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:58.571797 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 236/477] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:58.577282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9b862-3f53-4d5d-9c16-700e7a09739d None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:58.597085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c5c9f-dbcf-4cc7-935f-1ef0168e91f0 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:58.643017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf48fdd-6d68-40ac-9e09-2bc604bdf33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:58.643779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf48fdd-6d68-40ac-9e09-2bc604bdf33f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:58.644067 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf48fdd-6d68-40ac-9e09-2bc604bdf33f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:58.644347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf48fdd-6d68-40ac-9e09-2bc604bdf33f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:58.658883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf48fdd-6d68-40ac-9e09-2bc604bdf33f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-8cf48fdd-6d68-40ac-9e09-2bc604bdf33f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:42:58.668474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:42:58.669574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:42:58.670415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:42:58.670415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:42:58.670415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:42:58.670415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:42:58.670415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:42:58.670415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:42:58.670415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:42:58.670415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:42:58.670415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:42:58.670415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:42:58.670415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 18:42:58.670415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:42:58.670415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 242/478] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:42:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:42:58.678455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf48fdd-6d68-40ac-9e09-2bc604bdf33f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:42:58.716577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9b862-3f53-4d5d-9c16-700e7a09739d None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:58.732088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:42:58.758419 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e50d1-6c38-455c-99e2-47df4f0e6e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:58.759120 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e50d1-6c38-455c-99e2-47df4f0e6e4b None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:58.759410 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e50d1-6c38-455c-99e2-47df4f0e6e4b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:58.759688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e50d1-6c38-455c-99e2-47df4f0e6e4b None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:58.760807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e50d1-6c38-455c-99e2-47df4f0e6e4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:58.762177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e50d1-6c38-455c-99e2-47df4f0e6e4b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:58.804479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf48fdd-6d68-40ac-9e09-2bc604bdf33f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:42:58.846888 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2148a6-d89d-457d-8e08-50535643840c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:42:58.848825 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2148a6-d89d-457d-8e08-50535643840c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:58.849328 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2148a6-d89d-457d-8e08-50535643840c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:58.850197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2148a6-d89d-457d-8e08-50535643840c None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:58.851228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2148a6-d89d-457d-8e08-50535643840c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:42:58.851922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2148a6-d89d-457d-8e08-50535643840c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:42:58.853671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2148a6-d89d-457d-8e08-50535643840c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:42:58.864337 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 243/479] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:42:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:42:58.882241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 237/480] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:42:58 2022] POST /identity/v3/users => generated 249 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:42:59.944553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481d433-6e39-4c80-b1e4-ce445b91bcc0 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:59.945778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481d433-6e39-4c80-b1e4-ce445b91bcc0 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:59.946268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481d433-6e39-4c80-b1e4-ce445b91bcc0 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:59.948361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 244/481] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:42:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:42:59.954677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfaea0-143e-4fa7-a1cf-67c07a861a34 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:42:59.955649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfaea0-143e-4fa7-a1cf-67c07a861a34 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:42:59.956140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfaea0-143e-4fa7-a1cf-67c07a861a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:42:59.977286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-d3bfaea0-143e-4fa7-a1cf-67c07a861a34 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:42:59.978848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d3bfaea0-143e-4fa7-a1cf-67c07a861a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:42:59.989712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:00.057382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 238/482] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:00.082090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481d433-6e39-4c80-b1e4-ce445b91bcc0 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:00.245953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481d433-6e39-4c80-b1e4-ce445b91bcc0 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:00.250216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:00.311440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3c7d1-aef4-4b17-acfd-f00ebb17e4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:00.312943 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3c7d1-aef4-4b17-acfd-f00ebb17e4b8 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:00.313454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3c7d1-aef4-4b17-acfd-f00ebb17e4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:00.313962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3c7d1-aef4-4b17-acfd-f00ebb17e4b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:00.333894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3c7d1-aef4-4b17-acfd-f00ebb17e4b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-57f3c7d1-aef4-4b17-acfd-f00ebb17e4b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:00.350749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:00.351733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:00.352526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:00.352526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:00.352526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:00.352526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:00.352526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:00.352526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:00.352526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:00.352526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:00.352526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:00.352526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:00.352526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:00.352526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:00.353682 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 245/483] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:00.363278 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bfaea0-143e-4fa7-a1cf-67c07a861a34 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:00.558626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bfaea0-143e-4fa7-a1cf-67c07a861a34 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:00.620775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246a661-77d7-4efe-883a-9cde71e32250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:00.621897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246a661-77d7-4efe-883a-9cde71e32250 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:00.622460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246a661-77d7-4efe-883a-9cde71e32250 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:00.622985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246a661-77d7-4efe-883a-9cde71e32250 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:00.624453 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246a661-77d7-4efe-883a-9cde71e32250 None admin] RBAC: Adding query filter params (name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:00.625024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246a661-77d7-4efe-883a-9cde71e32250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:00.627688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246a661-77d7-4efe-883a-9cde71e32250 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:00.644272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 239/484] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:00.650154 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3c7d1-aef4-4b17-acfd-f00ebb17e4b8 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:00.717538 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadede5-992f-4b7e-8536-954326201da8 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:00.717863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadede5-992f-4b7e-8536-954326201da8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:00.718119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadede5-992f-4b7e-8536-954326201da8 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:00.719892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 240/485] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:00.737066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dae0e6-10b5-4ec6-9cbb-ce11ddfd272e None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:00.737572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dae0e6-10b5-4ec6-9cbb-ce11ddfd272e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:00.738117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dae0e6-10b5-4ec6-9cbb-ce11ddfd272e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:00.770714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:00.788132 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-26dae0e6-10b5-4ec6-9cbb-ce11ddfd272e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:00.789337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-26dae0e6-10b5-4ec6-9cbb-ce11ddfd272e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:00.858555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3c7d1-aef4-4b17-acfd-f00ebb17e4b8 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:00.925429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561eaad4-3144-4e0a-87c8-e82c52ffee9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:00.926252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561eaad4-3144-4e0a-87c8-e82c52ffee9f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:00.926593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561eaad4-3144-4e0a-87c8-e82c52ffee9f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:00.926874 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561eaad4-3144-4e0a-87c8-e82c52ffee9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:00.936457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 241/486] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:00.944147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561eaad4-3144-4e0a-87c8-e82c52ffee9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:00.945723 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561eaad4-3144-4e0a-87c8-e82c52ffee9f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-561eaad4-3144-4e0a-87c8-e82c52ffee9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:00.963958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:00.964865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:00.965793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:00.965793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:00.965793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:00.965793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:00.966001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 246/487] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:00.971845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dae0e6-10b5-4ec6-9cbb-ce11ddfd272e None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:00.996515 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561eaad4-3144-4e0a-87c8-e82c52ffee9f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:01.036576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:01.224765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561eaad4-3144-4e0a-87c8-e82c52ffee9f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:01.228096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dae0e6-10b5-4ec6-9cbb-ce11ddfd272e None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:01.274606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f4334-4621-4f76-9c62-54865d75c645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:01.275462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f4334-4621-4f76-9c62-54865d75c645 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:01.275778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f4334-4621-4f76-9c62-54865d75c645 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:01.276034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f4334-4621-4f76-9c62-54865d75c645 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:01.278631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c7298-8799-4e4a-93e9-fbee34da53fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:01.279752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c7298-8799-4e4a-93e9-fbee34da53fe None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:01.280277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c7298-8799-4e4a-93e9-fbee34da53fe None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:01.280769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c7298-8799-4e4a-93e9-fbee34da53fe None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:01.281695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c7298-8799-4e4a-93e9-fbee34da53fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:01.282166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c7298-8799-4e4a-93e9-fbee34da53fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:01.283734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c7298-8799-4e4a-93e9-fbee34da53fe None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:01.286714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:01.289522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f4334-4621-4f76-9c62-54865d75c645 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-ec6f4334-4621-4f76-9c62-54865d75c645 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:01.299348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:01.300729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:01.301563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:01.301563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:01.301563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:01.301563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:01.301563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:01.301563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:01.301563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:01.301563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:01.301563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:01.301563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:01.301563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:01.301563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:01.306537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 247/489] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:01.306537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b62f3-0f70-4013-b139-070edd1ccb9a None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:01.306537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b62f3-0f70-4013-b139-070edd1ccb9a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:01.306537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b62f3-0f70-4013-b139-070edd1ccb9a None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:01.307030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 242/489] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:01.308224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c7298-8799-4e4a-93e9-fbee34da53fe None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:01.311497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 248/490] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:01.312404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b62f3-0f70-4013-b139-070edd1ccb9a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:01.442738 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:01.447087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c7298-8799-4e4a-93e9-fbee34da53fe None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:01.475639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b62f3-0f70-4013-b139-070edd1ccb9a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:01.501206 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159562c-2a38-4d96-80ea-6f00054dc79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:01.502308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159562c-2a38-4d96-80ea-6f00054dc79d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:01.502766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159562c-2a38-4d96-80ea-6f00054dc79d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:01.503196 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159562c-2a38-4d96-80ea-6f00054dc79d None admin] PATH_INFO: `/v3/users/nova` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:01.514477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159562c-2a38-4d96-80ea-6f00054dc79d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:01.516486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159562c-2a38-4d96-80ea-6f00054dc79d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:01.539192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7a169-df3e-40bc-bd3f-344e067fdfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:01.539917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7a169-df3e-40bc-bd3f-344e067fdfd2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:01.540208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7a169-df3e-40bc-bd3f-344e067fdfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:01.540490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7a169-df3e-40bc-bd3f-344e067fdfd2 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:01.541404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7a169-df3e-40bc-bd3f-344e067fdfd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:01.541720 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7a169-df3e-40bc-bd3f-344e067fdfd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:01.543266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7a169-df3e-40bc-bd3f-344e067fdfd2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-2159562c-2a38-4d96-80ea-6f00054dc79d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:01.551857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:01.553966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:01.555513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:01.555513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:01.555513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:01.555513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:01.555513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:01.555513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:01.555513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:01.555513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:01.555513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:01.555513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:01.556487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 249/491] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:01.556487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ae69e-59d8-4aee-8af8-2891a9147480 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:01.556487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ae69e-59d8-4aee-8af8-2891a9147480 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:01.556487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ae69e-59d8-4aee-8af8-2891a9147480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:01.559464 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 243/492] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 18:43:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:01.565084 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159562c-2a38-4d96-80ea-6f00054dc79d None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:01.576039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-aa8ae69e-59d8-4aee-8af8-2891a9147480 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:01.577281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-aa8ae69e-59d8-4aee-8af8-2891a9147480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:01.649666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:01.661321 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 250/493] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:01.663444 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ae69e-59d8-4aee-8af8-2891a9147480 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:01.707825 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159562c-2a38-4d96-80ea-6f00054dc79d None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:01.759122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:01.773276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36633690-94cc-4e8e-8dfb-9f922c2e1c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:01.774460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36633690-94cc-4e8e-8dfb-9f922c2e1c0b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:01.774988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36633690-94cc-4e8e-8dfb-9f922c2e1c0b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:01.775443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36633690-94cc-4e8e-8dfb-9f922c2e1c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:01.795945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36633690-94cc-4e8e-8dfb-9f922c2e1c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:01.797145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ae69e-59d8-4aee-8af8-2891a9147480 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:01.798652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36633690-94cc-4e8e-8dfb-9f922c2e1c0b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-36633690-94cc-4e8e-8dfb-9f922c2e1c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:01.833170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:01.835143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:01.836000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:01.836000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:01.836000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:01.836000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:01.836208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 244/494] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:01.838899 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a6123-b644-49f2-9fda-bf65177b7c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:01.839594 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a6123-b644-49f2-9fda-bf65177b7c95 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:01.839886 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a6123-b644-49f2-9fda-bf65177b7c95 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:01.840169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a6123-b644-49f2-9fda-bf65177b7c95 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:01.840758 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af4b1c-472d-47e5-87d2-b1ba9ae16e7a None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:01.841029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a6123-b644-49f2-9fda-bf65177b7c95 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:01.841344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a6123-b644-49f2-9fda-bf65177b7c95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:01.841692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af4b1c-472d-47e5-87d2-b1ba9ae16e7a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:01.842173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af4b1c-472d-47e5-87d2-b1ba9ae16e7a None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:01.842955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a6123-b644-49f2-9fda-bf65177b7c95 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:01.844013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 245/495] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:01.844972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af4b1c-472d-47e5-87d2-b1ba9ae16e7a None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:01.885595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 251/496] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 18:43:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:01.886562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a6123-b644-49f2-9fda-bf65177b7c95 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:01.972368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af4b1c-472d-47e5-87d2-b1ba9ae16e7a None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:01.982054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:01.996033 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a6123-b644-49f2-9fda-bf65177b7c95 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:02.008029 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2c5b9-0109-4781-a1e3-cdc0c3c35037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:02.008871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2c5b9-0109-4781-a1e3-cdc0c3c35037 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.009150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2c5b9-0109-4781-a1e3-cdc0c3c35037 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.009422 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2c5b9-0109-4781-a1e3-cdc0c3c35037 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.020735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2c5b9-0109-4781-a1e3-cdc0c3c35037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:02.024308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-24c2c5b9-0109-4781-a1e3-cdc0c3c35037 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:02.028594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:02.029458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:02.030270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:02.030270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:02.030270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:02.030270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:02.030270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:02.030270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:02.030270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:02.030270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:02.030270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:02.030270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:02.030270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:02.030270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:02.030270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 246/497] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:02.031171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab796ca-8880-473a-886b-188d1e94c440 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.031525 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab796ca-8880-473a-886b-188d1e94c440 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.031815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab796ca-8880-473a-886b-188d1e94c440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.041314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a300447-f887-4846-948d-c4184da91432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:02.042006 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a300447-f887-4846-948d-c4184da91432 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.042294 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a300447-f887-4846-948d-c4184da91432 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.042608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a300447-f887-4846-948d-c4184da91432 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.043347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a300447-f887-4846-948d-c4184da91432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:02.043667 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a300447-f887-4846-948d-c4184da91432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:02.045053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a300447-f887-4846-948d-c4184da91432 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:02.050573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-eab796ca-8880-473a-886b-188d1e94c440 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:02.051806 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-eab796ca-8880-473a-886b-188d1e94c440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:02.059142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 252/498] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:02.060524 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04da1d-15f0-41e3-8fb8-bfcdbd101d81 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.060789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04da1d-15f0-41e3-8fb8-bfcdbd101d81 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.061071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04da1d-15f0-41e3-8fb8-bfcdbd101d81 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.066360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 253/499] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:02.066360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04da1d-15f0-41e3-8fb8-bfcdbd101d81 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:02.144402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 247/500] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:02.145299 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab796ca-8880-473a-886b-188d1e94c440 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:02.199297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04da1d-15f0-41e3-8fb8-bfcdbd101d81 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:02.242616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b38d4-af7a-40aa-8efc-d7c94124aab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:02.243631 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b38d4-af7a-40aa-8efc-d7c94124aab7 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.244168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b38d4-af7a-40aa-8efc-d7c94124aab7 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.244555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b38d4-af7a-40aa-8efc-d7c94124aab7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.267487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:02.271483 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab796ca-8880-473a-886b-188d1e94c440 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:02.305856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:02.314467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13dc9c-2b74-4456-9fd2-7941b07d67a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:02.315165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13dc9c-2b74-4456-9fd2-7941b07d67a7 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.315447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13dc9c-2b74-4456-9fd2-7941b07d67a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.315718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13dc9c-2b74-4456-9fd2-7941b07d67a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.316520 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13dc9c-2b74-4456-9fd2-7941b07d67a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:02.316828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13dc9c-2b74-4456-9fd2-7941b07d67a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:02.318216 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13dc9c-2b74-4456-9fd2-7941b07d67a7 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:02.320533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b38d4-af7a-40aa-8efc-d7c94124aab7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:02.322023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b38d4-af7a-40aa-8efc-d7c94124aab7 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:02.329630 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 248/501] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:02.330074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13dc9c-2b74-4456-9fd2-7941b07d67a7 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:02.434614 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 254/502] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:02.435923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2101725-9cf5-47b4-bc8e-a44935e866eb None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.436218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2101725-9cf5-47b4-bc8e-a44935e866eb None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.436504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2101725-9cf5-47b4-bc8e-a44935e866eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.445574 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13dc9c-2b74-4456-9fd2-7941b07d67a7 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:02.455878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-f2101725-9cf5-47b4-bc8e-a44935e866eb None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:02.457092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f2101725-9cf5-47b4-bc8e-a44935e866eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:02.485446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831d4f5-561b-44d8-8c59-817c55f0167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:02.486267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831d4f5-561b-44d8-8c59-817c55f0167f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.486604 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831d4f5-561b-44d8-8c59-817c55f0167f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.486858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831d4f5-561b-44d8-8c59-817c55f0167f None admin] PATH_INFO: `/v3/users/glance` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.496692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831d4f5-561b-44d8-8c59-817c55f0167f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:02.498064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831d4f5-561b-44d8-8c59-817c55f0167f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-2831d4f5-561b-44d8-8c59-817c55f0167f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:02.513102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:02.514061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:02.515018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:02.515018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:02.515018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:02.515018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:02.515018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:02.515018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:02.515018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:02.515018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:02.515018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 18:43:02.515018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:02.515018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 249/503] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:02.515655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831d4f5-561b-44d8-8c59-817c55f0167f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:02.534600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 255/504] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:02.535516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2101725-9cf5-47b4-bc8e-a44935e866eb None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:02.552528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:02.553615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:02.641937 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831d4f5-561b-44d8-8c59-817c55f0167f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:02.644524 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2101725-9cf5-47b4-bc8e-a44935e866eb None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:02.690074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651900cd-d1fa-4ea9-a12d-d9949a3bd292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:02.690914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651900cd-d1fa-4ea9-a12d-d9949a3bd292 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.691205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651900cd-d1fa-4ea9-a12d-d9949a3bd292 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.691486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651900cd-d1fa-4ea9-a12d-d9949a3bd292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.694510 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f142ed-c534-4253-9887-b8e25d92f9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:02.695721 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f142ed-c534-4253-9887-b8e25d92f9eb None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.696005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f142ed-c534-4253-9887-b8e25d92f9eb None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.696279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f142ed-c534-4253-9887-b8e25d92f9eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.703254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651900cd-d1fa-4ea9-a12d-d9949a3bd292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:02.704793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651900cd-d1fa-4ea9-a12d-d9949a3bd292 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:02.707051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f142ed-c534-4253-9887-b8e25d92f9eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:02.708491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f142ed-c534-4253-9887-b8e25d92f9eb None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-651900cd-d1fa-4ea9-a12d-d9949a3bd292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:02.723249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:02.724600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:02.725731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:02.725731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:02.725731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:02.725731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:02.725731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 256/505] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:02.725997 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651900cd-d1fa-4ea9-a12d-d9949a3bd292 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-a7f142ed-c534-4253-9887-b8e25d92f9eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:02.727768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:02.728577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:02.729414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:02.729414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:02.729414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:02.729414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:02.729414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 250/506] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:02.730162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f142ed-c534-4253-9887-b8e25d92f9eb None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:02.815256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:02.818660 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:02.851636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f142ed-c534-4253-9887-b8e25d92f9eb None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:02.851875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651900cd-d1fa-4ea9-a12d-d9949a3bd292 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:02.888996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9a1c0-ff5e-416e-9fdd-64dc95b18be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:02.889481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12659da-5843-4fd1-9102-79fab052cb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:02.889800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9a1c0-ff5e-416e-9fdd-64dc95b18be2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.890089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9a1c0-ff5e-416e-9fdd-64dc95b18be2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.890175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12659da-5843-4fd1-9102-79fab052cb9e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:02.890399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9a1c0-ff5e-416e-9fdd-64dc95b18be2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.890505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12659da-5843-4fd1-9102-79fab052cb9e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:02.890747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12659da-5843-4fd1-9102-79fab052cb9e None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:02.891592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12659da-5843-4fd1-9102-79fab052cb9e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:02.891906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12659da-5843-4fd1-9102-79fab052cb9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:02.893530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12659da-5843-4fd1-9102-79fab052cb9e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:02.901907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9a1c0-ff5e-416e-9fdd-64dc95b18be2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:02.903445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9a1c0-ff5e-416e-9fdd-64dc95b18be2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-5bb9a1c0-ff5e-416e-9fdd-64dc95b18be2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:02.923973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:02.925110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:02.926733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:02.926733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:02.926733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:02.926733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:02.926733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 257/507] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:02.926733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9a1c0-ff5e-416e-9fdd-64dc95b18be2 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:02.935998 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 251/508] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Jul 25 18:43:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:02.936869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12659da-5843-4fd1-9102-79fab052cb9e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.052875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12659da-5843-4fd1-9102-79fab052cb9e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.054178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9a1c0-ff5e-416e-9fdd-64dc95b18be2 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.076219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.086500 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.090677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad834f7d-4c73-4df6-b1bd-a9e723e524c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.091502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad834f7d-4c73-4df6-b1bd-a9e723e524c7 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.095859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad834f7d-4c73-4df6-b1bd-a9e723e524c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.095859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad834f7d-4c73-4df6-b1bd-a9e723e524c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.096050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6e404-c48a-496c-93e3-27f1d7ff4895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.096050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6e404-c48a-496c-93e3-27f1d7ff4895 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.096050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6e404-c48a-496c-93e3-27f1d7ff4895 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.096050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6e404-c48a-496c-93e3-27f1d7ff4895 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.096325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6e404-c48a-496c-93e3-27f1d7ff4895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:03.096325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6e404-c48a-496c-93e3-27f1d7ff4895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:03.097014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6e404-c48a-496c-93e3-27f1d7ff4895 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:03.104456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad834f7d-4c73-4df6-b1bd-a9e723e524c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:03.110988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 252/509] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:03.111769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6e404-c48a-496c-93e3-27f1d7ff4895 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-ad834f7d-4c73-4df6-b1bd-a9e723e524c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:03.112516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:03.113378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:03.114171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:03.114171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:03.114171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:03.114171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:03.114171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:03.114171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:03.114171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:03.114171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:03.114171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:03.114171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:03.114171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:03.114171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:03.114171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 258/510] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:03.121170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad834f7d-4c73-4df6-b1bd-a9e723e524c7 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.226576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6e404-c48a-496c-93e3-27f1d7ff4895 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.236591 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad834f7d-4c73-4df6-b1bd-a9e723e524c7 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.264739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc2f73-353a-4b67-8108-c7f7ee7e8bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.265447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc2f73-353a-4b67-8108-c7f7ee7e8bbd None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.265732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc2f73-353a-4b67-8108-c7f7ee7e8bbd None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.266017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc2f73-353a-4b67-8108-c7f7ee7e8bbd None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.266794 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc2f73-353a-4b67-8108-c7f7ee7e8bbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:03.267110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc2f73-353a-4b67-8108-c7f7ee7e8bbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:03.268530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc2f73-353a-4b67-8108-c7f7ee7e8bbd None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:03.283577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 253/511] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:03.284428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc2f73-353a-4b67-8108-c7f7ee7e8bbd None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.286821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23c17-e066-449f-9ed8-4df05dfc9aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.287650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23c17-e066-449f-9ed8-4df05dfc9aa1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.288207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23c17-e066-449f-9ed8-4df05dfc9aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.288558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23c17-e066-449f-9ed8-4df05dfc9aa1 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.289467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b23c17-e066-449f-9ed8-4df05dfc9aa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:03.289785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b23c17-e066-449f-9ed8-4df05dfc9aa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:03.291248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b23c17-e066-449f-9ed8-4df05dfc9aa1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:03.304504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 259/512] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:03.305489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23c17-e066-449f-9ed8-4df05dfc9aa1 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.344905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.351332 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.403699 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc2f73-353a-4b67-8108-c7f7ee7e8bbd None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.413503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23c17-e066-449f-9ed8-4df05dfc9aa1 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.441631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e82a80-8bbd-454b-8949-5b52c6d2b1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.442290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e82a80-8bbd-454b-8949-5b52c6d2b1e2 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.442602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e82a80-8bbd-454b-8949-5b52c6d2b1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.442882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e82a80-8bbd-454b-8949-5b52c6d2b1e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.450388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81f4cd-7d3e-4d4e-acaf-b46d0d441fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.451239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81f4cd-7d3e-4d4e-acaf-b46d0d441fd4 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.451525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81f4cd-7d3e-4d4e-acaf-b46d0d441fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.451803 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81f4cd-7d3e-4d4e-acaf-b46d0d441fd4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.460636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81f4cd-7d3e-4d4e-acaf-b46d0d441fd4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:03.462503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81f4cd-7d3e-4d4e-acaf-b46d0d441fd4 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-9c81f4cd-7d3e-4d4e-acaf-b46d0d441fd4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:03.479974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:03.480940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:03.482360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:03.482360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:03.482360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:03.482360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:03.482360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:03.482360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:03.482360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:03.482360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:03.482360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:03.482360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:03.482360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 260/513] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:03.482360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81f4cd-7d3e-4d4e-acaf-b46d0d441fd4 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.531447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e82a80-8bbd-454b-8949-5b52c6d2b1e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:03.532910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e82a80-8bbd-454b-8949-5b52c6d2b1e2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:03.584173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.590835 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.593910 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81f4cd-7d3e-4d4e-acaf-b46d0d441fd4 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.633642 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0b333-a7b5-48bb-ad46-73224c46c6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.634250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0b333-a7b5-48bb-ad46-73224c46c6df None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.634569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0b333-a7b5-48bb-ad46-73224c46c6df None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.634851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0b333-a7b5-48bb-ad46-73224c46c6df None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.635602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0b333-a7b5-48bb-ad46-73224c46c6df None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:03.635915 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0b333-a7b5-48bb-ad46-73224c46c6df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:03.637317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0b333-a7b5-48bb-ad46-73224c46c6df None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:03.642287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 254/514] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:03.643137 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e82a80-8bbd-454b-8949-5b52c6d2b1e2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.648026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 261/515] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:03.648026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0b333-a7b5-48bb-ad46-73224c46c6df None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.752505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e82a80-8bbd-454b-8949-5b52c6d2b1e2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.758417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0b333-a7b5-48bb-ad46-73224c46c6df None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.792939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84c7d1-140f-4fe2-a730-043ef5fdb2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.793682 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84c7d1-140f-4fe2-a730-043ef5fdb2a2 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.793950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84c7d1-140f-4fe2-a730-043ef5fdb2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.794255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84c7d1-140f-4fe2-a730-043ef5fdb2a2 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.794829 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20665625-0f36-468f-8ad5-91dfe9588b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.795354 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84c7d1-140f-4fe2-a730-043ef5fdb2a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:03.795727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20665625-0f36-468f-8ad5-91dfe9588b67 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.795932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20665625-0f36-468f-8ad5-91dfe9588b67 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.796217 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20665625-0f36-468f-8ad5-91dfe9588b67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.796638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84c7d1-140f-4fe2-a730-043ef5fdb2a2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:03.803773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20665625-0f36-468f-8ad5-91dfe9588b67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:03.805256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20665625-0f36-468f-8ad5-91dfe9588b67 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-20665625-0f36-468f-8ad5-91dfe9588b67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:03.822074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:03.827138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:03.828343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:03.828343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:03.828343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:03.828343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:03.828343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:03.828343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:03.828343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:03.828343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:03.828343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 262/516] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:03.828343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20665625-0f36-468f-8ad5-91dfe9588b67 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.860189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.908235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 255/517] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:43:03 2022] POST /identity/v3/users => generated 247 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:03.909001 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84c7d1-140f-4fe2-a730-043ef5fdb2a2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:03.919177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:03.946993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20665625-0f36-468f-8ad5-91dfe9588b67 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:03.988927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509d89e-9199-4d3b-a7de-8119287bdc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:03.989636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509d89e-9199-4d3b-a7de-8119287bdc28 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:03.989928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509d89e-9199-4d3b-a7de-8119287bdc28 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:03.990629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509d89e-9199-4d3b-a7de-8119287bdc28 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:03.991582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509d89e-9199-4d3b-a7de-8119287bdc28 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:03.991942 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509d89e-9199-4d3b-a7de-8119287bdc28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:03.993479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509d89e-9199-4d3b-a7de-8119287bdc28 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.018434 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 263/518] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:04.018434 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509d89e-9199-4d3b-a7de-8119287bdc28 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.042032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84c7d1-140f-4fe2-a730-043ef5fdb2a2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.094266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa470dd-aec0-4949-93c2-55874083b8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.095438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa470dd-aec0-4949-93c2-55874083b8b3 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.095903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa470dd-aec0-4949-93c2-55874083b8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.096358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa470dd-aec0-4949-93c2-55874083b8b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.118589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa470dd-aec0-4949-93c2-55874083b8b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:04.120963 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa470dd-aec0-4949-93c2-55874083b8b3 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.127242 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-daa470dd-aec0-4949-93c2-55874083b8b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:04.148671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:04.149793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:04.150883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:04.150883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:04.150883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:04.150883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:04.150883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 256/519] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:04.151224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa470dd-aec0-4949-93c2-55874083b8b3 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.156730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509d89e-9199-4d3b-a7de-8119287bdc28 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.201297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cd602-4d89-43c6-85f9-0551b78bca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.202111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cd602-4d89-43c6-85f9-0551b78bca6c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.202904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cd602-4d89-43c6-85f9-0551b78bca6c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.203193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cd602-4d89-43c6-85f9-0551b78bca6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.216226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cd602-4d89-43c6-85f9-0551b78bca6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:04.217825 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cd602-4d89-43c6-85f9-0551b78bca6c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.226976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-1e5cd602-4d89-43c6-85f9-0551b78bca6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:04.237989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:04.239310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:04.242879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:04.242879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:04.242879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:04.242879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:04.242879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 264/520] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:04.242879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cd602-4d89-43c6-85f9-0551b78bca6c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.305687 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa470dd-aec0-4949-93c2-55874083b8b3 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.358374 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b7143-2fcf-4da7-b72c-d178b7dc5386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.359059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b7143-2fcf-4da7-b72c-d178b7dc5386 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.359346 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b7143-2fcf-4da7-b72c-d178b7dc5386 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.359625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b7143-2fcf-4da7-b72c-d178b7dc5386 None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.360597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b7143-2fcf-4da7-b72c-d178b7dc5386 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:04.360769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b7143-2fcf-4da7-b72c-d178b7dc5386 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:04.362800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b7143-2fcf-4da7-b72c-d178b7dc5386 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.388983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 257/521] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:04.388983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b7143-2fcf-4da7-b72c-d178b7dc5386 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.400448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cd602-4d89-43c6-85f9-0551b78bca6c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.450387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:04.457976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064edb5-5f0e-4303-b9db-5755d963503d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.458971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064edb5-5f0e-4303-b9db-5755d963503d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.471116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064edb5-5f0e-4303-b9db-5755d963503d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.471116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064edb5-5f0e-4303-b9db-5755d963503d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.557537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b7143-2fcf-4da7-b72c-d178b7dc5386 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.571281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064edb5-5f0e-4303-b9db-5755d963503d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:04.573612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064edb5-5f0e-4303-b9db-5755d963503d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.590002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:04.615243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db0ed6-818e-4f3a-8c4c-9e922a1894ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.616031 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db0ed6-818e-4f3a-8c4c-9e922a1894ed None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.616261 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db0ed6-818e-4f3a-8c4c-9e922a1894ed None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.616546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db0ed6-818e-4f3a-8c4c-9e922a1894ed None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.617337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db0ed6-818e-4f3a-8c4c-9e922a1894ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:04.617789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db0ed6-818e-4f3a-8c4c-9e922a1894ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:04.619316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db0ed6-818e-4f3a-8c4c-9e922a1894ed None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.638723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 258/522] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:04.638723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db0ed6-818e-4f3a-8c4c-9e922a1894ed None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.710416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 265/523] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:04.711259 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064edb5-5f0e-4303-b9db-5755d963503d None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.749222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:04.766981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db0ed6-818e-4f3a-8c4c-9e922a1894ed None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.812142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2ebf6-1cdf-49fc-817c-d181eb687263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.812866 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2ebf6-1cdf-49fc-817c-d181eb687263 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.823403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2ebf6-1cdf-49fc-817c-d181eb687263 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.826013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2ebf6-1cdf-49fc-817c-d181eb687263 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.826429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2ebf6-1cdf-49fc-817c-d181eb687263 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:04.826752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2ebf6-1cdf-49fc-817c-d181eb687263 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:04.828637 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2ebf6-1cdf-49fc-817c-d181eb687263 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.836756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064edb5-5f0e-4303-b9db-5755d963503d None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:04.853233 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 259/524] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:04.853233 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2ebf6-1cdf-49fc-817c-d181eb687263 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.877439 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09ce8f-4007-485b-baa8-15755713fa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:04.878121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09ce8f-4007-485b-baa8-15755713fa68 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:04.878462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09ce8f-4007-485b-baa8-15755713fa68 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:04.878764 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09ce8f-4007-485b-baa8-15755713fa68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:04.879630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09ce8f-4007-485b-baa8-15755713fa68 None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=e628cb75e0c74f27a390945c8a04335c) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:04.879947 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09ce8f-4007-485b-baa8-15755713fa68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:04.881407 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09ce8f-4007-485b-baa8-15755713fa68 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:04.888535 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 266/525] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/role_assignments?user.id=e628cb75e0c74f27a390945c8a04335c&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:04.889252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09ce8f-4007-485b-baa8-15755713fa68 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:04.911156 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:05.008623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2ebf6-1cdf-49fc-817c-d181eb687263 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.036298 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:05.047430 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09ce8f-4007-485b-baa8-15755713fa68 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.052762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac63bac-6415-4bf7-8036-b56ac4ad4f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.054132 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac63bac-6415-4bf7-8036-b56ac4ad4f80 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.054581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac63bac-6415-4bf7-8036-b56ac4ad4f80 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.054937 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac63bac-6415-4bf7-8036-b56ac4ad4f80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.064117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac63bac-6415-4bf7-8036-b56ac4ad4f80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.066838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac63bac-6415-4bf7-8036-b56ac4ad4f80 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-4ac63bac-6415-4bf7-8036-b56ac4ad4f80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:05.088149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:05.089053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:05.089986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:05.089986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:05.089986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:05.089986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:05.089986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:05.089986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:05.089986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:05.089986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:05.097671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 260/526] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:05.097671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac63bac-6415-4bf7-8036-b56ac4ad4f80 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.107314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec22ad-f9e4-4ef1-a3b8-32ee51c786eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.108248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec22ad-f9e4-4ef1-a3b8-32ee51c786eb None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.108542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec22ad-f9e4-4ef1-a3b8-32ee51c786eb None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.108828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec22ad-f9e4-4ef1-a3b8-32ee51c786eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.122428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec22ad-f9e4-4ef1-a3b8-32ee51c786eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.123850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec22ad-f9e4-4ef1-a3b8-32ee51c786eb None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-83ec22ad-f9e4-4ef1-a3b8-32ee51c786eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:05.145531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:05.146967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:05.147795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:05.147795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:05.147795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:05.147795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:05.152026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 267/527] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:05.153065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec22ad-f9e4-4ef1-a3b8-32ee51c786eb None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.232126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:05.236372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac63bac-6415-4bf7-8036-b56ac4ad4f80 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.278286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199a8d5-f75d-4499-9917-eb356ade62ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.279389 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199a8d5-f75d-4499-9917-eb356ade62ce None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.279833 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199a8d5-f75d-4499-9917-eb356ade62ce None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.280227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199a8d5-f75d-4499-9917-eb356ade62ce None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.284252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec22ad-f9e4-4ef1-a3b8-32ee51c786eb None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.288755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199a8d5-f75d-4499-9917-eb356ade62ce None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.290512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199a8d5-f75d-4499-9917-eb356ade62ce None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-8199a8d5-f75d-4499-9917-eb356ade62ce None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:05.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:05.307522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:05.308372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:05.308372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:05.308372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:05.308372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:05.308372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:05.308372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:05.308372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:05.308372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:05.308372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 18:43:05.308372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:05.308372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 261/528] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:05.308913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199a8d5-f75d-4499-9917-eb356ade62ce None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.335384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39177ec-4409-460e-8731-ca0568d4512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.336401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39177ec-4409-460e-8731-ca0568d4512d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.336862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39177ec-4409-460e-8731-ca0568d4512d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.337312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39177ec-4409-460e-8731-ca0568d4512d None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.338651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39177ec-4409-460e-8731-ca0568d4512d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:05.339143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39177ec-4409-460e-8731-ca0568d4512d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.340863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39177ec-4409-460e-8731-ca0568d4512d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.366732 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 268/529] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:05.368051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39177ec-4409-460e-8731-ca0568d4512d None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.380631 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:05.432531 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199a8d5-f75d-4499-9917-eb356ade62ce None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.486801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c6e4e-0ad3-4155-bb0b-fdfc48ff7601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.488233 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c6e4e-0ad3-4155-bb0b-fdfc48ff7601 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.488551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c6e4e-0ad3-4155-bb0b-fdfc48ff7601 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.488836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c6e4e-0ad3-4155-bb0b-fdfc48ff7601 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.489876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c6e4e-0ad3-4155-bb0b-fdfc48ff7601 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:05.490192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c6e4e-0ad3-4155-bb0b-fdfc48ff7601 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.491853 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39177ec-4409-460e-8731-ca0568d4512d None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.492491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c6e4e-0ad3-4155-bb0b-fdfc48ff7601 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.510863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 262/530] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:05.512199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c6e4e-0ad3-4155-bb0b-fdfc48ff7601 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.536986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:05.537507 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd34cf3-95ff-4f38-a4c0-52d953487dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.538498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd34cf3-95ff-4f38-a4c0-52d953487dca None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.539250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd34cf3-95ff-4f38-a4c0-52d953487dca None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.539731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd34cf3-95ff-4f38-a4c0-52d953487dca None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.541711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd34cf3-95ff-4f38-a4c0-52d953487dca None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:05.542342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd34cf3-95ff-4f38-a4c0-52d953487dca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.544011 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd34cf3-95ff-4f38-a4c0-52d953487dca None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.565797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 269/531] 10.222.0.17 () {56 vars in 1139 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:05.567246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd34cf3-95ff-4f38-a4c0-52d953487dca None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.651514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c6e4e-0ad3-4155-bb0b-fdfc48ff7601 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.653111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:05.696748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd34cf3-95ff-4f38-a4c0-52d953487dca None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.701260 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c904a-a0d0-49e3-9ae9-23578af9d800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.702148 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c904a-a0d0-49e3-9ae9-23578af9d800 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.702622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c904a-a0d0-49e3-9ae9-23578af9d800 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.703044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c904a-a0d0-49e3-9ae9-23578af9d800 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.704028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c904a-a0d0-49e3-9ae9-23578af9d800 None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=28954d67689a46ee975fdfc9139e41eb) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:05.704491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c904a-a0d0-49e3-9ae9-23578af9d800 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.706639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c904a-a0d0-49e3-9ae9-23578af9d800 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.714304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 263/532] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/role_assignments?user.id=28954d67689a46ee975fdfc9139e41eb&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:05.715401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c904a-a0d0-49e3-9ae9-23578af9d800 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.791194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95d7f6-3a4d-403a-aa55-2ba29fc6df33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.792042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95d7f6-3a4d-403a-aa55-2ba29fc6df33 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.792393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95d7f6-3a4d-403a-aa55-2ba29fc6df33 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.792742 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95d7f6-3a4d-403a-aa55-2ba29fc6df33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:05.815661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95d7f6-3a4d-403a-aa55-2ba29fc6df33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:05.817456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95d7f6-3a4d-403a-aa55-2ba29fc6df33 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:05.829243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:05.843346 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c904a-a0d0-49e3-9ae9-23578af9d800 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-cb95d7f6-3a4d-403a-aa55-2ba29fc6df33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:05.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:05.848364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:05.849482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:05.849482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:05.849482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:05.849482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:05.849482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:05.849482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:05.849482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:05.849482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:05.849482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 270/533] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:05.859604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95d7f6-3a4d-403a-aa55-2ba29fc6df33 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:05.902644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ee9ee-0aa0-4c0b-88ea-3aecb5154a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:05.903335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ee9ee-0aa0-4c0b-88ea-3aecb5154a13 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:05.903622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ee9ee-0aa0-4c0b-88ea-3aecb5154a13 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:05.903898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ee9ee-0aa0-4c0b-88ea-3aecb5154a13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.036274 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ee9ee-0aa0-4c0b-88ea-3aecb5154a13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:06.039309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ee9ee-0aa0-4c0b-88ea-3aecb5154a13 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:06.060112 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95d7f6-3a4d-403a-aa55-2ba29fc6df33 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:06.126045 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:06.141381 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:06.149009 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1802b43-38c3-4ee3-ae3d-33cf60b6af05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:06.149719 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1802b43-38c3-4ee3-ae3d-33cf60b6af05 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.150012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1802b43-38c3-4ee3-ae3d-33cf60b6af05 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.150290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1802b43-38c3-4ee3-ae3d-33cf60b6af05 None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.151815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1802b43-38c3-4ee3-ae3d-33cf60b6af05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:06.152206 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1802b43-38c3-4ee3-ae3d-33cf60b6af05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:06.154211 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1802b43-38c3-4ee3-ae3d-33cf60b6af05 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:06.170154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 264/534] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:06.172488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d30112-32bb-464f-b7b2-7c5dc9a322a4 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.172927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 271/535] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:06.173247 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d30112-32bb-464f-b7b2-7c5dc9a322a4 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.173884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d30112-32bb-464f-b7b2-7c5dc9a322a4 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.178362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 265/536] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:06.179700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1802b43-38c3-4ee3-ae3d-33cf60b6af05 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:06.180964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d30112-32bb-464f-b7b2-7c5dc9a322a4 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:06.331099 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1802b43-38c3-4ee3-ae3d-33cf60b6af05 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:06.354179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d30112-32bb-464f-b7b2-7c5dc9a322a4 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:06.373944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73163d2a-dcd0-4236-a229-b2d4a8ab257b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:06.375146 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73163d2a-dcd0-4236-a229-b2d4a8ab257b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.375662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73163d2a-dcd0-4236-a229-b2d4a8ab257b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.376129 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73163d2a-dcd0-4236-a229-b2d4a8ab257b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.389380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73163d2a-dcd0-4236-a229-b2d4a8ab257b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:06.391239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73163d2a-dcd0-4236-a229-b2d4a8ab257b None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:06.403537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c1b8b-c651-4b94-8174-c9695e77bd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:06.404219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c1b8b-c651-4b94-8174-c9695e77bd52 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.404516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c1b8b-c651-4b94-8174-c9695e77bd52 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.404801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c1b8b-c651-4b94-8174-c9695e77bd52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.405857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c1b8b-c651-4b94-8174-c9695e77bd52 None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=9a419d02a7dc46e9b3c5b3d4eb3151d6) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:06.406269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c1b8b-c651-4b94-8174-c9695e77bd52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:06.407847 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c1b8b-c651-4b94-8174-c9695e77bd52 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:06.415130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 266/537] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/role_assignments?user.id=9a419d02a7dc46e9b3c5b3d4eb3151d6&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-73163d2a-dcd0-4236-a229-b2d4a8ab257b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:06.416063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:06.416913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:06.417731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:06.417731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:06.417731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:06.417731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:06.417948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d5962-cfb3-4c7b-9afa-cb557e542534 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.417948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d5962-cfb3-4c7b-9afa-cb557e542534 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.417948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d5962-cfb3-4c7b-9afa-cb557e542534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.420736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 272/538] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:06.430121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73163d2a-dcd0-4236-a229-b2d4a8ab257b None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:06.447873 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ff3d5962-cfb3-4c7b-9afa-cb557e542534 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:06.449115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ff3d5962-cfb3-4c7b-9afa-cb557e542534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:06.507655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:06.540785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 267/539] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:06.565869 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:06.592301 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d5962-cfb3-4c7b-9afa-cb557e542534 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:06.658055 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73163d2a-dcd0-4236-a229-b2d4a8ab257b None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:06.713256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65be77-0925-4027-a539-eebcb7512b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:06.713955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65be77-0925-4027-a539-eebcb7512b97 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.714252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65be77-0925-4027-a539-eebcb7512b97 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.714570 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65be77-0925-4027-a539-eebcb7512b97 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.715316 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65be77-0925-4027-a539-eebcb7512b97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:06.715624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65be77-0925-4027-a539-eebcb7512b97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:06.717130 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65be77-0925-4027-a539-eebcb7512b97 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:06.722238 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d5962-cfb3-4c7b-9afa-cb557e542534 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:06.738871 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 273/540] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:06.745911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65be77-0925-4027-a539-eebcb7512b97 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:06.768386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2c53d-5a6d-467b-b731-63d9c138301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:06.769207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2c53d-5a6d-467b-b731-63d9c138301e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.769505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2c53d-5a6d-467b-b731-63d9c138301e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.769785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2c53d-5a6d-467b-b731-63d9c138301e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.784442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2c53d-5a6d-467b-b731-63d9c138301e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-28e2c53d-5a6d-467b-b731-63d9c138301e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:06.792737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:06.793641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:06.794446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:06.794446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:06.794446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:06.794446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:06.794446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:06.794446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:06.794446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:06.794446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:06.794446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:06.794446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:06.794446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:06.794446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:06.795034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 268/541] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:06.801763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2c53d-5a6d-467b-b731-63d9c138301e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:06.908406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:06.915549 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65be77-0925-4027-a539-eebcb7512b97 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:06.924257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:06.958493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687224f9-d6e1-4d06-a3ca-66f734cc6471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:06.959932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687224f9-d6e1-4d06-a3ca-66f734cc6471 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:06.960275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687224f9-d6e1-4d06-a3ca-66f734cc6471 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:06.960568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687224f9-d6e1-4d06-a3ca-66f734cc6471 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:06.969153 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687224f9-d6e1-4d06-a3ca-66f734cc6471 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:06.970744 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687224f9-d6e1-4d06-a3ca-66f734cc6471 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-687224f9-d6e1-4d06-a3ca-66f734cc6471 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:06.989651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:06.991830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:06.993850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:06.993850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:06.993850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:06.993850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:06.993850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:06.993850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:06.993850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:06.993850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:06.993850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 274/542] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:06.999102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687224f9-d6e1-4d06-a3ca-66f734cc6471 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:07.098114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2c53d-5a6d-467b-b731-63d9c138301e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:07.158844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146dbcc-6752-4604-9067-e2bf3e105d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:07.159742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146dbcc-6752-4604-9067-e2bf3e105d68 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.159742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146dbcc-6752-4604-9067-e2bf3e105d68 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.160664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146dbcc-6752-4604-9067-e2bf3e105d68 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.160795 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146dbcc-6752-4604-9067-e2bf3e105d68 None admin] RBAC: Adding query filter params (name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:07.161173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146dbcc-6752-4604-9067-e2bf3e105d68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:07.163414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146dbcc-6752-4604-9067-e2bf3e105d68 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:07.184369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 269/543] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:07.188692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146dbcc-6752-4604-9067-e2bf3e105d68 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:07.222876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687224f9-d6e1-4d06-a3ca-66f734cc6471 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:07.294364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f6d6f-f14b-466d-82c6-226af005ab62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:07.294364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f6d6f-f14b-466d-82c6-226af005ab62 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.294364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f6d6f-f14b-466d-82c6-226af005ab62 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.294364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f6d6f-f14b-466d-82c6-226af005ab62 None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.294912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f6d6f-f14b-466d-82c6-226af005ab62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:07.294912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f6d6f-f14b-466d-82c6-226af005ab62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:07.294912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f6d6f-f14b-466d-82c6-226af005ab62 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:07.307607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 275/544] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:07.316714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f6d6f-f14b-466d-82c6-226af005ab62 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:07.397382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:07.464526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:07.470443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146dbcc-6752-4604-9067-e2bf3e105d68 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:07.509719 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f6d6f-f14b-466d-82c6-226af005ab62 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:07.521727 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795de8d-8e89-4fc6-b4bf-333ee9e7e31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:07.523351 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795de8d-8e89-4fc6-b4bf-333ee9e7e31a None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.523641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795de8d-8e89-4fc6-b4bf-333ee9e7e31a None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.523948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795de8d-8e89-4fc6-b4bf-333ee9e7e31a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.540102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795de8d-8e89-4fc6-b4bf-333ee9e7e31a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:07.542572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795de8d-8e89-4fc6-b4bf-333ee9e7e31a None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:07.551038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394aa22-1cf3-49f6-8676-b3fb316d9dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:07.551707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394aa22-1cf3-49f6-8676-b3fb316d9dec None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.551993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394aa22-1cf3-49f6-8676-b3fb316d9dec None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.552267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394aa22-1cf3-49f6-8676-b3fb316d9dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.553177 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394aa22-1cf3-49f6-8676-b3fb316d9dec None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=ffcf280bc6d54b2fb2d22634468ac6cd) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:07.554065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394aa22-1cf3-49f6-8676-b3fb316d9dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:07.555739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394aa22-1cf3-49f6-8676-b3fb316d9dec None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:07.562740 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 276/545] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:07 2022] GET /identity/v3/role_assignments?user.id=ffcf280bc6d54b2fb2d22634468ac6cd&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:07.564231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93325171-2404-4036-a1be-520158310d93 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.564531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93325171-2404-4036-a1be-520158310d93 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.564817 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93325171-2404-4036-a1be-520158310d93 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.566487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 277/546] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-1795de8d-8e89-4fc6-b4bf-333ee9e7e31a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:07.570431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:07.571508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:07.573122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:07.573122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:07.573122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:07.573122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:07.574216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90f81f-fccb-4d25-a60c-71e00e58112c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.574637 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90f81f-fccb-4d25-a60c-71e00e58112c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.575013 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90f81f-fccb-4d25-a60c-71e00e58112c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.575721 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 270/547] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:07.581821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795de8d-8e89-4fc6-b4bf-333ee9e7e31a None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:07.606124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-db90f81f-fccb-4d25-a60c-71e00e58112c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:07.609091 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-db90f81f-fccb-4d25-a60c-71e00e58112c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:07.766200 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 278/548] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:07.793960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90f81f-fccb-4d25-a60c-71e00e58112c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:07.809883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795de8d-8e89-4fc6-b4bf-333ee9e7e31a None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:07.815941 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:07.898452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c15515-5309-47b1-9030-9b99de08a5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:07.899255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c15515-5309-47b1-9030-9b99de08a5f2 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:07.899553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c15515-5309-47b1-9030-9b99de08a5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:07.899841 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c15515-5309-47b1-9030-9b99de08a5f2 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:07.900588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c15515-5309-47b1-9030-9b99de08a5f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:07.900910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c15515-5309-47b1-9030-9b99de08a5f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:07.904377 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c15515-5309-47b1-9030-9b99de08a5f2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:07.924473 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 271/549] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:07.930831 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c15515-5309-47b1-9030-9b99de08a5f2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:07.955289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:07.972847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90f81f-fccb-4d25-a60c-71e00e58112c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:08.031987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef03e9-13d7-4994-b163-026ef09b9f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:08.032736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef03e9-13d7-4994-b163-026ef09b9f89 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.033020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef03e9-13d7-4994-b163-026ef09b9f89 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.033295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef03e9-13d7-4994-b163-026ef09b9f89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.057469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef03e9-13d7-4994-b163-026ef09b9f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-a8ef03e9-13d7-4994-b163-026ef09b9f89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:08.075582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:08.076717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:08.077695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:08.077695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:08.077695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:08.077695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:08.077695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:08.077695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:08.077695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:08.077695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:08.077695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:08.077695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:08.077695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:08.077695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:08.077695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 279/550] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:08.087507 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef03e9-13d7-4994-b163-026ef09b9f89 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:08.130118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c15515-5309-47b1-9030-9b99de08a5f2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:08.193225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d6650-ec7e-4b16-8a03-8671f3ad70a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:08.195177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d6650-ec7e-4b16-8a03-8671f3ad70a7 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.196886 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d6650-ec7e-4b16-8a03-8671f3ad70a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.197322 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d6650-ec7e-4b16-8a03-8671f3ad70a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.215698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d6650-ec7e-4b16-8a03-8671f3ad70a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:08.217438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d6650-ec7e-4b16-8a03-8671f3ad70a7 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:08.232268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-fa3d6650-ec7e-4b16-8a03-8671f3ad70a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:08.242277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:08.243231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:08.244122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:08.244122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:08.244122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:08.244122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:08.244122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:08.244122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:08.244122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:08.244122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:08.244122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:08.244122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:08.251403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 272/551] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:08.253008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d6650-ec7e-4b16-8a03-8671f3ad70a7 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:08.283484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef03e9-13d7-4994-b163-026ef09b9f89 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:08.370171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955bdc3-4f9f-4f03-8f9c-1abe6b068282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:08.373004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955bdc3-4f9f-4f03-8f9c-1abe6b068282 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.373964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955bdc3-4f9f-4f03-8f9c-1abe6b068282 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.375017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955bdc3-4f9f-4f03-8f9c-1abe6b068282 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.376197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955bdc3-4f9f-4f03-8f9c-1abe6b068282 None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:08.376965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955bdc3-4f9f-4f03-8f9c-1abe6b068282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:08.384457 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955bdc3-4f9f-4f03-8f9c-1abe6b068282 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:08.413185 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 280/552] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:08.416589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77d343-761b-483d-b380-37124ba2ded6 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.418063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77d343-761b-483d-b380-37124ba2ded6 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.418984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77d343-761b-483d-b380-37124ba2ded6 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.421021 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 281/553] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:08.421803 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77d343-761b-483d-b380-37124ba2ded6 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:08.429186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:08.476773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d6650-ec7e-4b16-8a03-8671f3ad70a7 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:08.530052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3e76f-1e02-40fd-ac47-d0be44310852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:08.531787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3e76f-1e02-40fd-ac47-d0be44310852 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.532267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3e76f-1e02-40fd-ac47-d0be44310852 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.532747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3e76f-1e02-40fd-ac47-d0be44310852 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.534100 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3e76f-1e02-40fd-ac47-d0be44310852 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:08.534636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3e76f-1e02-40fd-ac47-d0be44310852 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:08.537158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3e76f-1e02-40fd-ac47-d0be44310852 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:08.561583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77d343-761b-483d-b380-37124ba2ded6 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:08.567173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 273/554] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Jul 25 18:43:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:08.569955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d396e-efa4-4c82-9e8c-3183c02c3989 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.570584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d396e-efa4-4c82-9e8c-3183c02c3989 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.571076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d396e-efa4-4c82-9e8c-3183c02c3989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.595685 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-011d396e-efa4-4c82-9e8c-3183c02c3989 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:08.597969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-011d396e-efa4-4c82-9e8c-3183c02c3989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:08.611563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091666fa-c37d-4371-8d97-96b203934b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:08.613435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091666fa-c37d-4371-8d97-96b203934b72 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.613937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091666fa-c37d-4371-8d97-96b203934b72 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.614937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091666fa-c37d-4371-8d97-96b203934b72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.618124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:08.632230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091666fa-c37d-4371-8d97-96b203934b72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:08.634665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091666fa-c37d-4371-8d97-96b203934b72 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-091666fa-c37d-4371-8d97-96b203934b72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:08.659751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:08.660769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:08.661654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:08.661654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:08.661654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:08.661654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:08.670615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 282/555] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:08.671579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091666fa-c37d-4371-8d97-96b203934b72 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:08.702750 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 274/556] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:08.703763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d396e-efa4-4c82-9e8c-3183c02c3989 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:08.801437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:08.816417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091666fa-c37d-4371-8d97-96b203934b72 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:08.860495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d396e-efa4-4c82-9e8c-3183c02c3989 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:08.868312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6dee3-a61f-4ba3-a459-ae1bde8d5d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:08.869877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6dee3-a61f-4ba3-a459-ae1bde8d5d6a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.870426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6dee3-a61f-4ba3-a459-ae1bde8d5d6a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.870910 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6dee3-a61f-4ba3-a459-ae1bde8d5d6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.902045 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e864c0-1dfc-4c4b-a55b-040829cc6768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:08.902883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e864c0-1dfc-4c4b-a55b-040829cc6768 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:08.903219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e864c0-1dfc-4c4b-a55b-040829cc6768 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:08.903503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e864c0-1dfc-4c4b-a55b-040829cc6768 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:08.904376 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e864c0-1dfc-4c4b-a55b-040829cc6768 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:08.904689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e864c0-1dfc-4c4b-a55b-040829cc6768 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:08.906358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e864c0-1dfc-4c4b-a55b-040829cc6768 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:08.927508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 275/557] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:08.929530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e864c0-1dfc-4c4b-a55b-040829cc6768 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:08.942644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:08.957621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6dee3-a61f-4ba3-a459-ae1bde8d5d6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:08.959709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6dee3-a61f-4ba3-a459-ae1bde8d5d6a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.049038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e864c0-1dfc-4c4b-a55b-040829cc6768 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.071416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:09.074555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 283/558] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:09.075875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6dee3-a61f-4ba3-a459-ae1bde8d5d6a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.093364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e720b45-91e2-4196-83ad-09b561bbad9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.094491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e720b45-91e2-4196-83ad-09b561bbad9f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.094922 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e720b45-91e2-4196-83ad-09b561bbad9f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.095342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e720b45-91e2-4196-83ad-09b561bbad9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.112496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e720b45-91e2-4196-83ad-09b561bbad9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-9e720b45-91e2-4196-83ad-09b561bbad9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:09.126460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:09.127884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:09.128761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:09.128761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:09.128761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:09.128761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:09.128761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:09.128761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:09.128761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:09.128761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:09.128761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:09.128761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:09.128761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:09.128761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:09.137190 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 276/559] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:09.139785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e720b45-91e2-4196-83ad-09b561bbad9f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.183181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:09.207651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6dee3-a61f-4ba3-a459-ae1bde8d5d6a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.260386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e94b50-411c-44f2-8aa1-2a4b3e83bed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.261755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e94b50-411c-44f2-8aa1-2a4b3e83bed8 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.262246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e94b50-411c-44f2-8aa1-2a4b3e83bed8 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.262796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e94b50-411c-44f2-8aa1-2a4b3e83bed8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.263286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e720b45-91e2-4196-83ad-09b561bbad9f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.278035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e94b50-411c-44f2-8aa1-2a4b3e83bed8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:09.280350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e94b50-411c-44f2-8aa1-2a4b3e83bed8 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-e0e94b50-411c-44f2-8aa1-2a4b3e83bed8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:09.304202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:09.305498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:09.306701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:09.306701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:09.306701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:09.306701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:09.306701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:09.306701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:09.306701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:09.306701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:09.306701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:09.306701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:09.306701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 284/560] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:09.307508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e792a-b6c6-4168-9a5a-38fc02dfd982 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.307508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e792a-b6c6-4168-9a5a-38fc02dfd982 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.307508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e792a-b6c6-4168-9a5a-38fc02dfd982 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.312860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 285/561] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:09.313696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e792a-b6c6-4168-9a5a-38fc02dfd982 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.316498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02231f8e-546d-4cf7-863d-2f9bf8b76765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.317349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02231f8e-546d-4cf7-863d-2f9bf8b76765 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.317670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02231f8e-546d-4cf7-863d-2f9bf8b76765 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.317950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02231f8e-546d-4cf7-863d-2f9bf8b76765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.332164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02231f8e-546d-4cf7-863d-2f9bf8b76765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:09.333556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02231f8e-546d-4cf7-863d-2f9bf8b76765 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-02231f8e-546d-4cf7-863d-2f9bf8b76765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:09.353644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:09.354789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:09.356225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:09.356225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:09.356225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:09.356225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:09.356225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 277/562] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:09.360385 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02231f8e-546d-4cf7-863d-2f9bf8b76765 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.398371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:09.455733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e792a-b6c6-4168-9a5a-38fc02dfd982 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.503718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02231f8e-546d-4cf7-863d-2f9bf8b76765 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.504832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed56c6-97fe-46d9-a5fe-6c2808e75507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.505513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed56c6-97fe-46d9-a5fe-6c2808e75507 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.505797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed56c6-97fe-46d9-a5fe-6c2808e75507 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.506077 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed56c6-97fe-46d9-a5fe-6c2808e75507 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.508188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed56c6-97fe-46d9-a5fe-6c2808e75507 None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:09.508188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed56c6-97fe-46d9-a5fe-6c2808e75507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:09.509168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed56c6-97fe-46d9-a5fe-6c2808e75507 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.520246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 286/563] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:09.521504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ae60-5fd8-47ee-a5a0-a237dca19888 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.521798 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ae60-5fd8-47ee-a5a0-a237dca19888 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.522110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ae60-5fd8-47ee-a5a0-a237dca19888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.544785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-bcd8ae60-5fd8-47ee-a5a0-a237dca19888 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:09.546014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-bcd8ae60-5fd8-47ee-a5a0-a237dca19888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:09.548389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:09.549278 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bf03d-2915-44db-b03c-44b43849f4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.549962 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bf03d-2915-44db-b03c-44b43849f4cd None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.550247 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bf03d-2915-44db-b03c-44b43849f4cd None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.550561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bf03d-2915-44db-b03c-44b43849f4cd None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.551508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644bf03d-2915-44db-b03c-44b43849f4cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:09.551950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644bf03d-2915-44db-b03c-44b43849f4cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:09.555120 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644bf03d-2915-44db-b03c-44b43849f4cd None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.577557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 278/564] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:09.578193 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bf03d-2915-44db-b03c-44b43849f4cd None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.662454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 287/565] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:09.662454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8ae60-5fd8-47ee-a5a0-a237dca19888 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.698521 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:09.714589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bf03d-2915-44db-b03c-44b43849f4cd None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.780996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccf769-3e14-42bf-a63d-b5ba16763159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.781689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccf769-3e14-42bf-a63d-b5ba16763159 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.781978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccf769-3e14-42bf-a63d-b5ba16763159 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.782260 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccf769-3e14-42bf-a63d-b5ba16763159 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.783029 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ccf769-3e14-42bf-a63d-b5ba16763159 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:09.783338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ccf769-3e14-42bf-a63d-b5ba16763159 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:09.785089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ccf769-3e14-42bf-a63d-b5ba16763159 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.800920 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 279/566] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:09.802578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccf769-3e14-42bf-a63d-b5ba16763159 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.823122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8ae60-5fd8-47ee-a5a0-a237dca19888 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.861428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff551d17-ab34-452d-9922-c5a63528e0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.862246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff551d17-ab34-452d-9922-c5a63528e0c2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.862564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff551d17-ab34-452d-9922-c5a63528e0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.862910 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff551d17-ab34-452d-9922-c5a63528e0c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.870545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:09.874674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff551d17-ab34-452d-9922-c5a63528e0c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:09.876151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff551d17-ab34-452d-9922-c5a63528e0c2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-ff551d17-ab34-452d-9922-c5a63528e0c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:09.895796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:09.902827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:09.903911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:09.903911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:09.903911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:09.903911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:09.903911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 288/567] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:09.903911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff551d17-ab34-452d-9922-c5a63528e0c2 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:09.919051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccf769-3e14-42bf-a63d-b5ba16763159 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:09.955399 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20a040-19da-4d1e-8c74-17fc56ae61fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:09.956211 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20a040-19da-4d1e-8c74-17fc56ae61fc None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:09.956496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20a040-19da-4d1e-8c74-17fc56ae61fc None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:09.956778 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20a040-19da-4d1e-8c74-17fc56ae61fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:09.969369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20a040-19da-4d1e-8c74-17fc56ae61fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:09.970893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20a040-19da-4d1e-8c74-17fc56ae61fc None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:09.979892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-8c20a040-19da-4d1e-8c74-17fc56ae61fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:09.990774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:09.999666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:10.000579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:10.000579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:10.000579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:10.000579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:10.000579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 280/568] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:10.000579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20a040-19da-4d1e-8c74-17fc56ae61fc None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.009697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff551d17-ab34-452d-9922-c5a63528e0c2 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.048449 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7325e-bf4c-4dbd-b441-ca6d64dd4a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.049285 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7325e-bf4c-4dbd-b441-ca6d64dd4a3f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.049577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7325e-bf4c-4dbd-b441-ca6d64dd4a3f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.049870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7325e-bf4c-4dbd-b441-ca6d64dd4a3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.063361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7325e-bf4c-4dbd-b441-ca6d64dd4a3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-75d7325e-bf4c-4dbd-b441-ca6d64dd4a3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:10.071966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:10.073788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 289/569] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:10.075301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359f686-efbf-47ca-a279-0ab70cbc64a8 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.076046 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359f686-efbf-47ca-a279-0ab70cbc64a8 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.076046 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359f686-efbf-47ca-a279-0ab70cbc64a8 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.078460 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 290/570] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:10.079533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359f686-efbf-47ca-a279-0ab70cbc64a8 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.111937 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20a040-19da-4d1e-8c74-17fc56ae61fc None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.143622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:10.164761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410d345-9453-46a1-8e2e-40c9d75ddffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.165569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410d345-9453-46a1-8e2e-40c9d75ddffc None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.165943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410d345-9453-46a1-8e2e-40c9d75ddffc None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.166133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410d345-9453-46a1-8e2e-40c9d75ddffc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.173625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410d345-9453-46a1-8e2e-40c9d75ddffc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:10.175270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410d345-9453-46a1-8e2e-40c9d75ddffc None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-0410d345-9453-46a1-8e2e-40c9d75ddffc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:10.191062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:10.192047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:10.193019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:10.193019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:10.193019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:10.193019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:10.193019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:10.193019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:10.193019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:10.193019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:10.193019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 281/571] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:10.193764 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410d345-9453-46a1-8e2e-40c9d75ddffc None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.213443 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359f686-efbf-47ca-a279-0ab70cbc64a8 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.255196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a6907-b48d-4a79-b7f2-411bdfc3b64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.255832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a6907-b48d-4a79-b7f2-411bdfc3b64f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.256170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a6907-b48d-4a79-b7f2-411bdfc3b64f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.256482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a6907-b48d-4a79-b7f2-411bdfc3b64f None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.257983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a6907-b48d-4a79-b7f2-411bdfc3b64f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:10.258442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a6907-b48d-4a79-b7f2-411bdfc3b64f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:10.259848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a6907-b48d-4a79-b7f2-411bdfc3b64f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:10.267320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:10.272917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 291/572] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:10.273962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a6907-b48d-4a79-b7f2-411bdfc3b64f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.311900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410d345-9453-46a1-8e2e-40c9d75ddffc None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.348179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21766e-3194-438d-a5a6-0ccedc9044b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.348863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21766e-3194-438d-a5a6-0ccedc9044b7 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.349425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21766e-3194-438d-a5a6-0ccedc9044b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.349705 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21766e-3194-438d-a5a6-0ccedc9044b7 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.350475 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21766e-3194-438d-a5a6-0ccedc9044b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:10.350996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21766e-3194-438d-a5a6-0ccedc9044b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:10.353077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21766e-3194-438d-a5a6-0ccedc9044b7 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:10.370497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 282/573] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:10.371968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3125af0-9947-4f1e-b078-c46c6c006416 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.372267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3125af0-9947-4f1e-b078-c46c6c006416 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.372551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3125af0-9947-4f1e-b078-c46c6c006416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.393159 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b3125af0-9947-4f1e-b078-c46c6c006416 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:10.394404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b3125af0-9947-4f1e-b078-c46c6c006416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:10.408046 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a6907-b48d-4a79-b7f2-411bdfc3b64f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.420461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:10.447906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9389c7b-4eb8-49d8-bfa3-1fa0e9d67cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.448595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9389c7b-4eb8-49d8-bfa3-1fa0e9d67cc3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.448912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9389c7b-4eb8-49d8-bfa3-1fa0e9d67cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.449150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9389c7b-4eb8-49d8-bfa3-1fa0e9d67cc3 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.449908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9389c7b-4eb8-49d8-bfa3-1fa0e9d67cc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:10.450228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9389c7b-4eb8-49d8-bfa3-1fa0e9d67cc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:10.451646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9389c7b-4eb8-49d8-bfa3-1fa0e9d67cc3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:10.460960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 292/574] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:10.461779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9389c7b-4eb8-49d8-bfa3-1fa0e9d67cc3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.472165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 283/575] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:10.473013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3125af0-9947-4f1e-b078-c46c6c006416 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.536814 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:10.587095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9389c7b-4eb8-49d8-bfa3-1fa0e9d67cc3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.591626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3125af0-9947-4f1e-b078-c46c6c006416 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.627275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c9034-4a29-41cc-8825-1786c3e58ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.628397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9034-4a29-41cc-8825-1786c3e58ea4 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.628813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9034-4a29-41cc-8825-1786c3e58ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.629095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9034-4a29-41cc-8825-1786c3e58ea4 None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.630241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c9034-4a29-41cc-8825-1786c3e58ea4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:10.630790 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c9034-4a29-41cc-8825-1786c3e58ea4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:10.630885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1bead-a543-4323-a0c6-b87e46dfe6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.632226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1bead-a543-4323-a0c6-b87e46dfe6cd None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.632581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1bead-a543-4323-a0c6-b87e46dfe6cd None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.632932 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1bead-a543-4323-a0c6-b87e46dfe6cd None admin] PATH_INFO: `/v3/users/glance` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.633893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c9034-4a29-41cc-8825-1786c3e58ea4 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:10.647972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 293/576] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:10.648801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c9034-4a29-41cc-8825-1786c3e58ea4 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.656982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1bead-a543-4323-a0c6-b87e46dfe6cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:10.659738 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1bead-a543-4323-a0c6-b87e46dfe6cd None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-40a1bead-a543-4323-a0c6-b87e46dfe6cd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:10.691341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:10.692913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:10.694694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:10.694694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:10.694694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:10.694694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:10.694694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:10.694694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:10.694694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:10.694694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:10.694694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 18:43:10.694694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:10.694694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 284/577] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:10.696057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1bead-a543-4323-a0c6-b87e46dfe6cd None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.713434 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:10.819738 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c9034-4a29-41cc-8825-1786c3e58ea4 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.842352 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1bead-a543-4323-a0c6-b87e46dfe6cd None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:10.860133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:10.869903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a17c65-b22a-45a5-8b2d-0f65a4cd40d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.872076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a17c65-b22a-45a5-8b2d-0f65a4cd40d3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.872076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a17c65-b22a-45a5-8b2d-0f65a4cd40d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.872076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a17c65-b22a-45a5-8b2d-0f65a4cd40d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.883005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a17c65-b22a-45a5-8b2d-0f65a4cd40d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:10.884485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a17c65-b22a-45a5-8b2d-0f65a4cd40d3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:10.897477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b6de6-df8b-41e6-9e3b-88c780303c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:10.898907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b6de6-df8b-41e6-9e3b-88c780303c06 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:10.899427 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b6de6-df8b-41e6-9e3b-88c780303c06 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:10.900166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b6de6-df8b-41e6-9e3b-88c780303c06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-05a17c65-b22a-45a5-8b2d-0f65a4cd40d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:10.900561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:10.901847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:10.903498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:10.903498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:10.903498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:10.903498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:10.903498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:10.903498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:10.903498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:10.903498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:10.906507 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 294/578] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:10.907926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a17c65-b22a-45a5-8b2d-0f65a4cd40d3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:10.927864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b6de6-df8b-41e6-9e3b-88c780303c06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:10.931130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b6de6-df8b-41e6-9e3b-88c780303c06 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-e79b6de6-df8b-41e6-9e3b-88c780303c06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:10.960607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:10.962142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:10.963357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:10.963357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:10.963357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:10.963357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:10.963357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 285/579] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:10.963357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b6de6-df8b-41e6-9e3b-88c780303c06 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:11.036534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:11.052682 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a17c65-b22a-45a5-8b2d-0f65a4cd40d3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.108194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99d837-a5d6-41fd-9f10-329429f4acb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.109423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99d837-a5d6-41fd-9f10-329429f4acb2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.109886 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99d837-a5d6-41fd-9f10-329429f4acb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.110255 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99d837-a5d6-41fd-9f10-329429f4acb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.133655 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99d837-a5d6-41fd-9f10-329429f4acb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:11.143154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b6de6-df8b-41e6-9e3b-88c780303c06 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-5e99d837-a5d6-41fd-9f10-329429f4acb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:11.143425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:11.145235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:11.147828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:11.147828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:11.147828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:11.147828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:11.147828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:11.147828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:11.147828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:11.147828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:11.147828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:11.147828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:11.147828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:11.147828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:11.149911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 295/580] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:11.152481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99d837-a5d6-41fd-9f10-329429f4acb2 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:11.205046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b9eb1-1930-48a8-ac28-b58b82457e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.206080 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b9eb1-1930-48a8-ac28-b58b82457e01 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.206551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b9eb1-1930-48a8-ac28-b58b82457e01 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.206838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b9eb1-1930-48a8-ac28-b58b82457e01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.207693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b9eb1-1930-48a8-ac28-b58b82457e01 None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=1ee9152572224cc499229e0fa2b81ac5) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:11.208014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b9eb1-1930-48a8-ac28-b58b82457e01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:11.209537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b9eb1-1930-48a8-ac28-b58b82457e01 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:11.218289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 286/581] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:10 2022] GET /identity/v3/role_assignments?user.id=1ee9152572224cc499229e0fa2b81ac5&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:11.218289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b9eb1-1930-48a8-ac28-b58b82457e01 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:11.272706 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:11.340649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99d837-a5d6-41fd-9f10-329429f4acb2 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.409188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213146b-bbb9-4e37-8ced-585a7ad53c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.409188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213146b-bbb9-4e37-8ced-585a7ad53c42 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.409188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213146b-bbb9-4e37-8ced-585a7ad53c42 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.410206 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213146b-bbb9-4e37-8ced-585a7ad53c42 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.410711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213146b-bbb9-4e37-8ced-585a7ad53c42 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:11.411198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213146b-bbb9-4e37-8ced-585a7ad53c42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:11.413045 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213146b-bbb9-4e37-8ced-585a7ad53c42 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:11.416482 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b9eb1-1930-48a8-ac28-b58b82457e01 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.444673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 296/582] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Jul 25 18:43:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:11.445656 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213146b-bbb9-4e37-8ced-585a7ad53c42 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:11.465661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:11.488786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f38e6-c5a4-4eb0-b8b4-1cc8a0739fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.489674 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f38e6-c5a4-4eb0-b8b4-1cc8a0739fc9 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.490057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f38e6-c5a4-4eb0-b8b4-1cc8a0739fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.490592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f38e6-c5a4-4eb0-b8b4-1cc8a0739fc9 None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.492064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f38e6-c5a4-4eb0-b8b4-1cc8a0739fc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:11.492386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f38e6-c5a4-4eb0-b8b4-1cc8a0739fc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:11.494155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f38e6-c5a4-4eb0-b8b4-1cc8a0739fc9 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:11.510658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 287/583] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:11.511532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f38e6-c5a4-4eb0-b8b4-1cc8a0739fc9 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:11.607813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213146b-bbb9-4e37-8ced-585a7ad53c42 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.645744 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:11.663528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb30f4-bec2-4ab5-ad63-8a2584b9e1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.668421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb30f4-bec2-4ab5-ad63-8a2584b9e1b0 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.668421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb30f4-bec2-4ab5-ad63-8a2584b9e1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.668421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb30f4-bec2-4ab5-ad63-8a2584b9e1b0 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.668421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb30f4-bec2-4ab5-ad63-8a2584b9e1b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:11.668421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb30f4-bec2-4ab5-ad63-8a2584b9e1b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:11.668421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb30f4-bec2-4ab5-ad63-8a2584b9e1b0 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:11.670190 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f38e6-c5a4-4eb0-b8b4-1cc8a0739fc9 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.686053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 297/584] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:11.686053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb30f4-bec2-4ab5-ad63-8a2584b9e1b0 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:11.739989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be0a7e-29be-4bb7-bbf8-7b31adec947b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.740692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be0a7e-29be-4bb7-bbf8-7b31adec947b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.741646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be0a7e-29be-4bb7-bbf8-7b31adec947b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.741892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be0a7e-29be-4bb7-bbf8-7b31adec947b None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.742759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be0a7e-29be-4bb7-bbf8-7b31adec947b None admin] RBAC: Adding query filter params (name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:11.743289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be0a7e-29be-4bb7-bbf8-7b31adec947b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:11.744804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be0a7e-29be-4bb7-bbf8-7b31adec947b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:11.759657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 288/585] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:11.759657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be0a7e-29be-4bb7-bbf8-7b31adec947b None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:11.784312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:11.832271 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb30f4-bec2-4ab5-ad63-8a2584b9e1b0 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.878351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff272c1c-6615-4d77-b65b-142c3d48903b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.879777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff272c1c-6615-4d77-b65b-142c3d48903b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.880356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff272c1c-6615-4d77-b65b-142c3d48903b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.880867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff272c1c-6615-4d77-b65b-142c3d48903b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.891433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be0a7e-29be-4bb7-bbf8-7b31adec947b None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:11.902800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff272c1c-6615-4d77-b65b-142c3d48903b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:11.905008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff272c1c-6615-4d77-b65b-142c3d48903b None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-ff272c1c-6615-4d77-b65b-142c3d48903b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:11.924906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:11.925844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:11.926867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:11.926867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:11.926867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:11.926867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:11.926867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 298/586] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:11.927148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff272c1c-6615-4d77-b65b-142c3d48903b None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:11.937433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb592e-f970-42ce-b252-0efb62249649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:11.938370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb592e-f970-42ce-b252-0efb62249649 None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:11.938661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb592e-f970-42ce-b252-0efb62249649 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:11.938938 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb592e-f970-42ce-b252-0efb62249649 None admin] PATH_INFO: `/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/e628cb75e0c74f27a390945c8a04335c/roles/1f308d13b8d04c0b9309a0916dee061f` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:11.960467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb592e-f970-42ce-b252-0efb62249649 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d7c3b139f284435aea4ff08a2f0952b, user_id=e628cb75e0c74f27a390945c8a04335c, role_id=1f308d13b8d04c0b9309a0916dee061f)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:11.962222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb592e-f970-42ce-b252-0efb62249649 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.049094 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:12.055613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 289/587] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:11 2022] PUT /identity/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/e628cb75e0c74f27a390945c8a04335c/roles/1f308d13b8d04c0b9309a0916dee061f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:12.056576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb592e-f970-42ce-b252-0efb62249649 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.070070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff272c1c-6615-4d77-b65b-142c3d48903b None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.104937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:12.126422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47be1f0-1565-420a-8424-4d586efb5eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.127939 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47be1f0-1565-420a-8424-4d586efb5eba None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.129633 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47be1f0-1565-420a-8424-4d586efb5eba None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.130235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47be1f0-1565-420a-8424-4d586efb5eba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.146648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47be1f0-1565-420a-8424-4d586efb5eba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:12.149553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47be1f0-1565-420a-8424-4d586efb5eba None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-e47be1f0-1565-420a-8424-4d586efb5eba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:12.170731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:12.172150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:12.173315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:12.173315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:12.173315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:12.173315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:12.173315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:12.173315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:12.173315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:12.173315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:12.173315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:12.173315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:12.178372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 299/588] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:12.178372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47be1f0-1565-420a-8424-4d586efb5eba None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.232470 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb592e-f970-42ce-b252-0efb62249649 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.287323 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d63ce-b276-4716-96f3-8005e1c03755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.288456 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d63ce-b276-4716-96f3-8005e1c03755 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.288661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d63ce-b276-4716-96f3-8005e1c03755 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.289049 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d63ce-b276-4716-96f3-8005e1c03755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.306498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d63ce-b276-4716-96f3-8005e1c03755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:12.308263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d63ce-b276-4716-96f3-8005e1c03755 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.329122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47be1f0-1565-420a-8424-4d586efb5eba None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-594d63ce-b276-4716-96f3-8005e1c03755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:12.329463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:12.330297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:12.331282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:12.331282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:12.331282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:12.331282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:12.333720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 290/589] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:12.335726 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d63ce-b276-4716-96f3-8005e1c03755 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.383455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c44c28-65de-4bf6-8f23-e95db9682070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.383455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c44c28-65de-4bf6-8f23-e95db9682070 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.383455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c44c28-65de-4bf6-8f23-e95db9682070 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.383455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c44c28-65de-4bf6-8f23-e95db9682070 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.384342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c44c28-65de-4bf6-8f23-e95db9682070 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:12.384342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c44c28-65de-4bf6-8f23-e95db9682070 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:12.384342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c44c28-65de-4bf6-8f23-e95db9682070 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.403146 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 300/590] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:12.404498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c44c28-65de-4bf6-8f23-e95db9682070 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.416196 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:12.463860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:12.515855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d63ce-b276-4716-96f3-8005e1c03755 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.534196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c44c28-65de-4bf6-8f23-e95db9682070 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.572580 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd994473-ec0e-4b8b-9330-3494a50ce799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.573270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd994473-ec0e-4b8b-9330-3494a50ce799 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.573586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd994473-ec0e-4b8b-9330-3494a50ce799 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.573858 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd994473-ec0e-4b8b-9330-3494a50ce799 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.574631 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd994473-ec0e-4b8b-9330-3494a50ce799 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:12.575044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd994473-ec0e-4b8b-9330-3494a50ce799 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:12.575143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f2595-f791-403d-83a1-40badd2d76bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.575844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f2595-f791-403d-83a1-40badd2d76bf None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.576133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f2595-f791-403d-83a1-40badd2d76bf None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.576413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f2595-f791-403d-83a1-40badd2d76bf None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.576655 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd994473-ec0e-4b8b-9330-3494a50ce799 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.577258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f2595-f791-403d-83a1-40badd2d76bf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:12.577572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f2595-f791-403d-83a1-40badd2d76bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:12.579059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f2595-f791-403d-83a1-40badd2d76bf None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.590358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 301/591] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:12.591308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd994473-ec0e-4b8b-9330-3494a50ce799 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.605818 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 291/592] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:12.606844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f2595-f791-403d-83a1-40badd2d76bf None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.711861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd994473-ec0e-4b8b-9330-3494a50ce799 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.713620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:12.724805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f2595-f791-403d-83a1-40badd2d76bf None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.735636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:12.748420 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59bca6-ac08-4e60-abae-fc837be53f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.749228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59bca6-ac08-4e60-abae-fc837be53f41 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.749594 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59bca6-ac08-4e60-abae-fc837be53f41 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.749974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59bca6-ac08-4e60-abae-fc837be53f41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.758642 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59bca6-ac08-4e60-abae-fc837be53f41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:12.760190 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59bca6-ac08-4e60-abae-fc837be53f41 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.766951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f933d-2a54-45e1-acef-60291420ff60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.767882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f933d-2a54-45e1-acef-60291420ff60 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.768174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f933d-2a54-45e1-acef-60291420ff60 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.768454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f933d-2a54-45e1-acef-60291420ff60 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-5b59bca6-ac08-4e60-abae-fc837be53f41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:12.775686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:12.776590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:12.778094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:12.778094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:12.778094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:12.778094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:12.778094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:12.778094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:12.778094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:12.778094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:12.778094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 302/593] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:12.778610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f933d-2a54-45e1-acef-60291420ff60 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:12.778694 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59bca6-ac08-4e60-abae-fc837be53f41 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.781408 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f933d-2a54-45e1-acef-60291420ff60 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-b47f933d-2a54-45e1-acef-60291420ff60 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:12.797438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:12.799739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:12.802198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:12.802198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:12.802198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:12.802198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:12.802198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:12.802198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:12.802198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:12.802198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:12.802198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 18:43:12.802198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:12.802198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 292/594] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:12.802198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f933d-2a54-45e1-acef-60291420ff60 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.887666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59bca6-ac08-4e60-abae-fc837be53f41 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.919606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f933d-2a54-45e1-acef-60291420ff60 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:12.925536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eba7c4-613d-4a0c-b5ae-b89b6a69074f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.926412 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eba7c4-613d-4a0c-b5ae-b89b6a69074f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.926708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eba7c4-613d-4a0c-b5ae-b89b6a69074f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.926995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eba7c4-613d-4a0c-b5ae-b89b6a69074f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.943450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eba7c4-613d-4a0c-b5ae-b89b6a69074f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:12.945798 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eba7c4-613d-4a0c-b5ae-b89b6a69074f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.964725 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1947d50-a028-4a34-830f-76df97d4dceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:12.965331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1947d50-a028-4a34-830f-76df97d4dceb None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:12.965620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1947d50-a028-4a34-830f-76df97d4dceb None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:12.965901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1947d50-a028-4a34-830f-76df97d4dceb None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:12.966864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1947d50-a028-4a34-830f-76df97d4dceb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-16eba7c4-613d-4a0c-b5ae-b89b6a69074f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:12.967061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:12.969322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:12.971584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:12.971584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:12.971584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:12.971584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:12.972086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1947d50-a028-4a34-830f-76df97d4dceb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:12.972240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 303/595] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:12.972240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eba7c4-613d-4a0c-b5ae-b89b6a69074f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:12.972528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1947d50-a028-4a34-830f-76df97d4dceb None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:12.991118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:12.992788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 293/596] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:12.993907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1947d50-a028-4a34-830f-76df97d4dceb None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.029297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:13.089425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eba7c4-613d-4a0c-b5ae-b89b6a69074f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.117836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1947d50-a028-4a34-830f-76df97d4dceb None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.126739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784669d-4048-43f4-b07e-d4ba3885af96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.127438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784669d-4048-43f4-b07e-d4ba3885af96 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.128145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784669d-4048-43f4-b07e-d4ba3885af96 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.128145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784669d-4048-43f4-b07e-d4ba3885af96 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.128850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784669d-4048-43f4-b07e-d4ba3885af96 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:13.129173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784669d-4048-43f4-b07e-d4ba3885af96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.130752 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784669d-4048-43f4-b07e-d4ba3885af96 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:13.153656 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 304/597] 10.222.0.17 () {56 vars in 1139 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:13.159822 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784669d-4048-43f4-b07e-d4ba3885af96 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.161992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446007d-453d-4e8e-be7a-be814e5ac4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.163069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446007d-453d-4e8e-be7a-be814e5ac4f9 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.163532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446007d-453d-4e8e-be7a-be814e5ac4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.164056 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446007d-453d-4e8e-be7a-be814e5ac4f9 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.164936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446007d-453d-4e8e-be7a-be814e5ac4f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:13.165395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446007d-453d-4e8e-be7a-be814e5ac4f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.166949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446007d-453d-4e8e-be7a-be814e5ac4f9 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:13.187322 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 294/598] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:13.188858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446007d-453d-4e8e-be7a-be814e5ac4f9 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.237390 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:13.269971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784669d-4048-43f4-b07e-d4ba3885af96 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.299943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:13.310517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdaee69-3840-444d-b421-0c158e935df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.311513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaee69-3840-444d-b421-0c158e935df6 None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.311763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaee69-3840-444d-b421-0c158e935df6 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.311991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaee69-3840-444d-b421-0c158e935df6 None admin] PATH_INFO: `/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/28954d67689a46ee975fdfc9139e41eb/roles/1f308d13b8d04c0b9309a0916dee061f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.319806 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446007d-453d-4e8e-be7a-be814e5ac4f9 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.331962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdaee69-3840-444d-b421-0c158e935df6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d7c3b139f284435aea4ff08a2f0952b, user_id=28954d67689a46ee975fdfc9139e41eb, role_id=1f308d13b8d04c0b9309a0916dee061f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.333441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdaee69-3840-444d-b421-0c158e935df6 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:13.365323 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167a742-3848-4060-8e10-2d04b03ebe10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.366715 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167a742-3848-4060-8e10-2d04b03ebe10 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.367058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167a742-3848-4060-8e10-2d04b03ebe10 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.367281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167a742-3848-4060-8e10-2d04b03ebe10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.382503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167a742-3848-4060-8e10-2d04b03ebe10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.384116 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167a742-3848-4060-8e10-2d04b03ebe10 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-f167a742-3848-4060-8e10-2d04b03ebe10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:13.412552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:13.416688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:13.418546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:13.418546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:13.418546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:13.418546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:13.418546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 295/599] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:13.418546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167a742-3848-4060-8e10-2d04b03ebe10 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.462037 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 305/600] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:13 2022] PUT /identity/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/28954d67689a46ee975fdfc9139e41eb/roles/1f308d13b8d04c0b9309a0916dee061f => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:13.466269 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdaee69-3840-444d-b421-0c158e935df6 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.595486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167a742-3848-4060-8e10-2d04b03ebe10 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.650918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d34ea-27b7-423f-8591-7464c8881ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.651770 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d34ea-27b7-423f-8591-7464c8881ee4 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.652060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d34ea-27b7-423f-8591-7464c8881ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.652342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d34ea-27b7-423f-8591-7464c8881ee4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.662311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d34ea-27b7-423f-8591-7464c8881ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.665052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d34ea-27b7-423f-8591-7464c8881ee4 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:13.668409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-479d34ea-27b7-423f-8591-7464c8881ee4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:13.686402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:13.687894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:13.689347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:13.689347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:13.689347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:13.689347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:13.689347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:13.689347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:13.689347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:13.689347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:13.689347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 296/601] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:13.690107 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7411ac-8425-4743-abe0-ff7e41c67c2f None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.690107 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7411ac-8425-4743-abe0-ff7e41c67c2f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.690306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7411ac-8425-4743-abe0-ff7e41c67c2f None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.696358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 297/602] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:13.699257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7411ac-8425-4743-abe0-ff7e41c67c2f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.703783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdaee69-3840-444d-b421-0c158e935df6 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.758678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:13.781573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af651da7-79f6-4963-b3f9-e03cf5b233aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.782344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af651da7-79f6-4963-b3f9-e03cf5b233aa None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.782650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af651da7-79f6-4963-b3f9-e03cf5b233aa None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.782929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af651da7-79f6-4963-b3f9-e03cf5b233aa None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.783730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af651da7-79f6-4963-b3f9-e03cf5b233aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:13.784040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af651da7-79f6-4963-b3f9-e03cf5b233aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.785522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af651da7-79f6-4963-b3f9-e03cf5b233aa None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:13.802093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 306/603] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:13.804142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942592a-5f45-457f-8e37-e2ced862fcb4 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.804592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942592a-5f45-457f-8e37-e2ced862fcb4 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.805095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942592a-5f45-457f-8e37-e2ced862fcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.831114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-7942592a-5f45-457f-8e37-e2ced862fcb4 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:13.832740 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7942592a-5f45-457f-8e37-e2ced862fcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:13.896502 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7411ac-8425-4743-abe0-ff7e41c67c2f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:13.938480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 307/604] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:13.940234 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942592a-5f45-457f-8e37-e2ced862fcb4 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:13.980224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa3674-14dc-4f15-b787-c7d3f95e1d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:13.981215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa3674-14dc-4f15-b787-c7d3f95e1d5d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:13.981505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa3674-14dc-4f15-b787-c7d3f95e1d5d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:13.981786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa3674-14dc-4f15-b787-c7d3f95e1d5d None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:13.983309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa3674-14dc-4f15-b787-c7d3f95e1d5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:13.983624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa3674-14dc-4f15-b787-c7d3f95e1d5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:13.985502 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa3674-14dc-4f15-b787-c7d3f95e1d5d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:14.004150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 298/605] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:14.005363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa3674-14dc-4f15-b787-c7d3f95e1d5d None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.104782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:14.144019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942592a-5f45-457f-8e37-e2ced862fcb4 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.175015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa3674-14dc-4f15-b787-c7d3f95e1d5d None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.192189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:14.233916 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0237cc7-d944-45a8-b9f5-a7992b91fe18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.235074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0237cc7-d944-45a8-b9f5-a7992b91fe18 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.235495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0237cc7-d944-45a8-b9f5-a7992b91fe18 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.236019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0237cc7-d944-45a8-b9f5-a7992b91fe18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.239392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f84d54-ae5d-40cd-819b-da4596831a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.240624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f84d54-ae5d-40cd-819b-da4596831a9c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.240827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f84d54-ae5d-40cd-819b-da4596831a9c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.242557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f84d54-ae5d-40cd-819b-da4596831a9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.252138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f84d54-ae5d-40cd-819b-da4596831a9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:14.253796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f84d54-ae5d-40cd-819b-da4596831a9c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:14.257373 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0237cc7-d944-45a8-b9f5-a7992b91fe18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-b0237cc7-d944-45a8-b9f5-a7992b91fe18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:14.266460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:14.267584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:14.268400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:14.268400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:14.268400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:14.268400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:14.268400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:14.268400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:14.268400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:14.268400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:14.268400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:14.268400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:14.268400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:14.268400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-99f84d54-ae5d-40cd-819b-da4596831a9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:14.270232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:14.271188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:14.272062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:14.272062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:14.272062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:14.272062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:14.272062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:14.272062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:14.272062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:14.272062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:14.272062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 308/606] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:14.279379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 299/607] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:14.280655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0237cc7-d944-45a8-b9f5-a7992b91fe18 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.284273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f84d54-ae5d-40cd-819b-da4596831a9c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.422091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0237cc7-d944-45a8-b9f5-a7992b91fe18 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.449536 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:14.468998 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b8c90-76b2-41ed-8829-dd4bd4c043ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.470192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b8c90-76b2-41ed-8829-dd4bd4c043ff None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.470731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b8c90-76b2-41ed-8829-dd4bd4c043ff None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.471358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b8c90-76b2-41ed-8829-dd4bd4c043ff None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.472519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b8c90-76b2-41ed-8829-dd4bd4c043ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:14.473144 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b8c90-76b2-41ed-8829-dd4bd4c043ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:14.475267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b8c90-76b2-41ed-8829-dd4bd4c043ff None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:14.504918 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f84d54-ae5d-40cd-819b-da4596831a9c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.509385 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 300/608] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:14.510803 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b8c90-76b2-41ed-8829-dd4bd4c043ff None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.556976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3a179-bfcb-4f49-a6b4-2b599ee0e2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.558107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3a179-bfcb-4f49-a6b4-2b599ee0e2ea None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.558543 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3a179-bfcb-4f49-a6b4-2b599ee0e2ea None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.558832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3a179-bfcb-4f49-a6b4-2b599ee0e2ea None admin] PATH_INFO: `/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/9a419d02a7dc46e9b3c5b3d4eb3151d6/roles/1f308d13b8d04c0b9309a0916dee061f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.578006 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3a179-bfcb-4f49-a6b4-2b599ee0e2ea None admin] RBAC: Authorizing `identity:create_grant(project_id=5d7c3b139f284435aea4ff08a2f0952b, user_id=9a419d02a7dc46e9b3c5b3d4eb3151d6, role_id=1f308d13b8d04c0b9309a0916dee061f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:14.579525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3a179-bfcb-4f49-a6b4-2b599ee0e2ea None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:14.640879 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b8c90-76b2-41ed-8829-dd4bd4c043ff None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.691579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63af27-8487-47b1-aae0-2d6f81bc21f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.692601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63af27-8487-47b1-aae0-2d6f81bc21f6 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.692983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63af27-8487-47b1-aae0-2d6f81bc21f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.693270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63af27-8487-47b1-aae0-2d6f81bc21f6 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.695749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63af27-8487-47b1-aae0-2d6f81bc21f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:14.696081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63af27-8487-47b1-aae0-2d6f81bc21f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:14.697650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63af27-8487-47b1-aae0-2d6f81bc21f6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:14.718700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 309/609] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:14 2022] PUT /identity/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/9a419d02a7dc46e9b3c5b3d4eb3151d6/roles/1f308d13b8d04c0b9309a0916dee061f => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:14.719110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 301/610] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:14.723551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63af27-8487-47b1-aae0-2d6f81bc21f6 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.726680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d70df-8f80-4114-85e2-79a8a67e39c7 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.727135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d70df-8f80-4114-85e2-79a8a67e39c7 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.727493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d70df-8f80-4114-85e2-79a8a67e39c7 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.729440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 310/611] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:14.730350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d70df-8f80-4114-85e2-79a8a67e39c7 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:14.796039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:14.812989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:14.906393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d70df-8f80-4114-85e2-79a8a67e39c7 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.916387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63af27-8487-47b1-aae0-2d6f81bc21f6 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:14.975978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eefb03-c052-4ea6-a62f-d283c70aa8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.976807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eefb03-c052-4ea6-a62f-d283c70aa8b9 None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.977043 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0c7fd-7ab7-4a65-8200-87f5d1c7660f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:14.977266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eefb03-c052-4ea6-a62f-d283c70aa8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.977553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eefb03-c052-4ea6-a62f-d283c70aa8b9 None admin] PATH_INFO: `/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/ffcf280bc6d54b2fb2d22634468ac6cd/roles/1f308d13b8d04c0b9309a0916dee061f` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.978561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0c7fd-7ab7-4a65-8200-87f5d1c7660f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:14.979056 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0c7fd-7ab7-4a65-8200-87f5d1c7660f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:14.979533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0c7fd-7ab7-4a65-8200-87f5d1c7660f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:14.997162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0c7fd-7ab7-4a65-8200-87f5d1c7660f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:14.998742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eefb03-c052-4ea6-a62f-d283c70aa8b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d7c3b139f284435aea4ff08a2f0952b, user_id=ffcf280bc6d54b2fb2d22634468ac6cd, role_id=1f308d13b8d04c0b9309a0916dee061f)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:14.999623 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0c7fd-7ab7-4a65-8200-87f5d1c7660f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:15.000509 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eefb03-c052-4ea6-a62f-d283c70aa8b9 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-8be0c7fd-7ab7-4a65-8200-87f5d1c7660f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:15.023166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:15.024237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:15.025106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:15.025106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:15.025106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:15.025106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:15.025106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 311/612] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:15.031329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb9ec4-4a90-4c32-99cf-2d0c92664a99 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:15.031638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb9ec4-4a90-4c32-99cf-2d0c92664a99 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:15.032148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb9ec4-4a90-4c32-99cf-2d0c92664a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:15.053292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d2fb9ec4-4a90-4c32-99cf-2d0c92664a99 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:15.055364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d2fb9ec4-4a90-4c32-99cf-2d0c92664a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:15.107423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 302/613] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:14 2022] PUT /identity/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/ffcf280bc6d54b2fb2d22634468ac6cd/roles/1f308d13b8d04c0b9309a0916dee061f => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:15.112961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eefb03-c052-4ea6-a62f-d283c70aa8b9 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:15.173897 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 312/614] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:15.205784 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb9ec4-4a90-4c32-99cf-2d0c92664a99 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:15.210974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:15.362764 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb9ec4-4a90-4c32-99cf-2d0c92664a99 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:15.388407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:15.412036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eefb03-c052-4ea6-a62f-d283c70aa8b9 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:15.416702 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734f823-318e-4470-af2f-2a9af891e558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:15.418935 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734f823-318e-4470-af2f-2a9af891e558 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:15.419561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734f823-318e-4470-af2f-2a9af891e558 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:15.420055 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734f823-318e-4470-af2f-2a9af891e558 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:15.436087 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734f823-318e-4470-af2f-2a9af891e558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-3734f823-318e-4470-af2f-2a9af891e558 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:15.448371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:15.449358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:15.450274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:15.450274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:15.450274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:15.450274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:15.450274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:15.450274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:15.450274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:15.450274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:15.450274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:15.450274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:15.450274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:15.450274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:15.456162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 313/615] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:15.461025 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734f823-318e-4470-af2f-2a9af891e558 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:15.492404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccfbfc-985e-41c8-8bf6-e17bebdf2963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:15.493680 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccfbfc-985e-41c8-8bf6-e17bebdf2963 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:15.494187 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccfbfc-985e-41c8-8bf6-e17bebdf2963 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:15.494700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccfbfc-985e-41c8-8bf6-e17bebdf2963 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:15.496668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccfbfc-985e-41c8-8bf6-e17bebdf2963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:15.496668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccfbfc-985e-41c8-8bf6-e17bebdf2963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:15.499294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccfbfc-985e-41c8-8bf6-e17bebdf2963 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:15.526403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 303/616] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:15.531762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccfbfc-985e-41c8-8bf6-e17bebdf2963 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:15.582659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:15.701640 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734f823-318e-4470-af2f-2a9af891e558 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:15.745872 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891008f9-b718-4f60-b6a3-3db5cdad0dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:15.746602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891008f9-b718-4f60-b6a3-3db5cdad0dda None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:15.746901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891008f9-b718-4f60-b6a3-3db5cdad0dda None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:15.747185 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891008f9-b718-4f60-b6a3-3db5cdad0dda None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:15.748331 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891008f9-b718-4f60-b6a3-3db5cdad0dda None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:15.748647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891008f9-b718-4f60-b6a3-3db5cdad0dda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:15.750039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891008f9-b718-4f60-b6a3-3db5cdad0dda None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:15.760968 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 314/617] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:15.770394 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891008f9-b718-4f60-b6a3-3db5cdad0dda None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:15.872566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccfbfc-985e-41c8-8bf6-e17bebdf2963 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:15.903789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891008f9-b718-4f60-b6a3-3db5cdad0dda None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:15.946842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:15.959129 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85818974-f077-47d0-8f31-486e152fa49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:15.961183 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85818974-f077-47d0-8f31-486e152fa49b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:15.961696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85818974-f077-47d0-8f31-486e152fa49b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:15.962200 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85818974-f077-47d0-8f31-486e152fa49b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:15.984421 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab43d35-1750-4ccc-8f31-88d46e17a31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:15.985505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab43d35-1750-4ccc-8f31-88d46e17a31b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:15.985804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab43d35-1750-4ccc-8f31-88d46e17a31b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:15.986228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab43d35-1750-4ccc-8f31-88d46e17a31b None admin] PATH_INFO: `/v3/users/swift` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:16.000725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85818974-f077-47d0-8f31-486e152fa49b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:16.002597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85818974-f077-47d0-8f31-486e152fa49b None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:16.004763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab43d35-1750-4ccc-8f31-88d46e17a31b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:16.008142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab43d35-1750-4ccc-8f31-88d46e17a31b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-85818974-f077-47d0-8f31-486e152fa49b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:16.036393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:16.037408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:16.038649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:16.038649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:16.038649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:16.038649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:16.041109 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 315/618] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:16.047095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85818974-f077-47d0-8f31-486e152fa49b None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-5ab43d35-1750-4ccc-8f31-88d46e17a31b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:16.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:16.054917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:16.056009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:16.056009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:16.056009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:16.056009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:16.056009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:16.056009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:16.056009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:16.056009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:16.056009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:16.056009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:16.059974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 304/619] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 525 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:16.061511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab43d35-1750-4ccc-8f31-88d46e17a31b None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:16.089599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:16.256339 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab43d35-1750-4ccc-8f31-88d46e17a31b None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:16.283530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85818974-f077-47d0-8f31-486e152fa49b None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:16.318701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f70fe6-1ee2-497a-8db6-97265dc10d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:16.320681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f70fe6-1ee2-497a-8db6-97265dc10d42 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:16.321480 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f70fe6-1ee2-497a-8db6-97265dc10d42 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:16.322238 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f70fe6-1ee2-497a-8db6-97265dc10d42 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:16.323940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f70fe6-1ee2-497a-8db6-97265dc10d42 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:16.325024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f70fe6-1ee2-497a-8db6-97265dc10d42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:16.327909 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f70fe6-1ee2-497a-8db6-97265dc10d42 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:16.329673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ed719-1801-4e8f-9662-a2625f54ed16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:16.330571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ed719-1801-4e8f-9662-a2625f54ed16 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:16.330972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ed719-1801-4e8f-9662-a2625f54ed16 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:16.331378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ed719-1801-4e8f-9662-a2625f54ed16 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:16.332339 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ed719-1801-4e8f-9662-a2625f54ed16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:16.332839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ed719-1801-4e8f-9662-a2625f54ed16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:16.335848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ed719-1801-4e8f-9662-a2625f54ed16 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:16.358313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 316/620] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:16.358313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fb927-cbf1-4323-a651-8757786ca5cd None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:16.358313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fb927-cbf1-4323-a651-8757786ca5cd None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:16.358313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fb927-cbf1-4323-a651-8757786ca5cd None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:16.362911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 317/621] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:16.364123 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fb927-cbf1-4323-a651-8757786ca5cd None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:16.366125 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 305/622] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Jul 25 18:43:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:16.368294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9b0f9-2e4e-4230-aede-befa8348b57c None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:16.368775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9b0f9-2e4e-4230-aede-befa8348b57c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:16.370262 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9b0f9-2e4e-4230-aede-befa8348b57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:16.398276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-d5d9b0f9-2e4e-4230-aede-befa8348b57c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:16.400893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d5d9b0f9-2e4e-4230-aede-befa8348b57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:16.431978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:16.550914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fb927-cbf1-4323-a651-8757786ca5cd None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:16.558383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:16.578398 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 306/623] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:16.578398 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d9b0f9-2e4e-4230-aede-befa8348b57c None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:16.655979 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f3a1e-9b0b-4c5b-9f14-8b37d99dc053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:16.657716 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f3a1e-9b0b-4c5b-9f14-8b37d99dc053 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:16.658228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f3a1e-9b0b-4c5b-9f14-8b37d99dc053 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:16.658794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f3a1e-9b0b-4c5b-9f14-8b37d99dc053 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:16.675418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f3a1e-9b0b-4c5b-9f14-8b37d99dc053 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:16.678193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f3a1e-9b0b-4c5b-9f14-8b37d99dc053 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-e86f3a1e-9b0b-4c5b-9f14-8b37d99dc053 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:16.710493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:16.712463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:16.715054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:16.715054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:16.715054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:16.715054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:16.715054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:16.715054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:16.715054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:16.715054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:16.715054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:16.715054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:16.715054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 318/624] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 18:43:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:16.716751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f3a1e-9b0b-4c5b-9f14-8b37d99dc053 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:16.725951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d9b0f9-2e4e-4230-aede-befa8348b57c None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:16.802312 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84b7af-316b-4d33-878e-fd15d28a8683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:16.804497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84b7af-316b-4d33-878e-fd15d28a8683 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:16.804954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84b7af-316b-4d33-878e-fd15d28a8683 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:16.805340 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84b7af-316b-4d33-878e-fd15d28a8683 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:16.821986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84b7af-316b-4d33-878e-fd15d28a8683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:16.824638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84b7af-316b-4d33-878e-fd15d28a8683 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-fe84b7af-316b-4d33-878e-fd15d28a8683 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:16.850976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:16.852386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:16.853834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:16.853834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:16.853834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:16.853834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:16.858455 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 307/625] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:16.860804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84b7af-316b-4d33-878e-fd15d28a8683 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:16.894492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:16.900957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f3a1e-9b0b-4c5b-9f14-8b37d99dc053 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:16.970426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:16.981842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba42b3-6565-497e-96ca-1c9c7ce7e1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:16.983632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba42b3-6565-497e-96ca-1c9c7ce7e1bd None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:16.984149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba42b3-6565-497e-96ca-1c9c7ce7e1bd None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:16.985241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba42b3-6565-497e-96ca-1c9c7ce7e1bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:17.011966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba42b3-6565-497e-96ca-1c9c7ce7e1bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-e9ba42b3-6565-497e-96ca-1c9c7ce7e1bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:17.032892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:17.034577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:17.035446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:17.035446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:17.035446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:17.035446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:17.035446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:17.035446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:17.035446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:17.035446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:17.035446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:17.035446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:17.035446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:17.035446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:17.037235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 319/626] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:17.041042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba42b3-6565-497e-96ca-1c9c7ce7e1bd None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:17.056154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84b7af-316b-4d33-878e-fd15d28a8683 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:17.100358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615812f-2424-4220-9427-c898ea413df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:17.101345 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615812f-2424-4220-9427-c898ea413df0 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:17.101640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615812f-2424-4220-9427-c898ea413df0 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:17.101927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615812f-2424-4220-9427-c898ea413df0 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:17.102917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615812f-2424-4220-9427-c898ea413df0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:17.103293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615812f-2424-4220-9427-c898ea413df0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:17.105243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615812f-2424-4220-9427-c898ea413df0 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:17.127875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 308/627] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 18:43:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:17.129019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615812f-2424-4220-9427-c898ea413df0 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:17.263626 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba42b3-6565-497e-96ca-1c9c7ce7e1bd None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:17.276657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615812f-2424-4220-9427-c898ea413df0 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:17.289618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:17.318839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1a95d-5580-4d20-8eb1-66dcf8057fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:17.320059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1a95d-5580-4d20-8eb1-66dcf8057fd2 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:17.320360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1a95d-5580-4d20-8eb1-66dcf8057fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:17.320644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1a95d-5580-4d20-8eb1-66dcf8057fd2 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:17.321795 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1a95d-5580-4d20-8eb1-66dcf8057fd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:17.322206 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1a95d-5580-4d20-8eb1-66dcf8057fd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:17.323666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1a95d-5580-4d20-8eb1-66dcf8057fd2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:17.336218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a4bc4-24a3-488f-90d7-90878fd1e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:17.337744 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a4bc4-24a3-488f-90d7-90878fd1e2dd None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:17.338483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a4bc4-24a3-488f-90d7-90878fd1e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:17.339186 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a4bc4-24a3-488f-90d7-90878fd1e2dd None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:17.340724 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 309/628] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:17.341049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a4bc4-24a3-488f-90d7-90878fd1e2dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:17.342478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1a95d-5580-4d20-8eb1-66dcf8057fd2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:17.342646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a4bc4-24a3-488f-90d7-90878fd1e2dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:17.344869 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a4bc4-24a3-488f-90d7-90878fd1e2dd None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:17.371206 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 320/629] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:17.373204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a4bc4-24a3-488f-90d7-90878fd1e2dd None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:17.412335 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:17.490931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1a95d-5580-4d20-8eb1-66dcf8057fd2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:17.539121 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e375959-8c5c-4da6-9bad-02a139d8a2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:17.539740 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e375959-8c5c-4da6-9bad-02a139d8a2bf None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:17.540031 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e375959-8c5c-4da6-9bad-02a139d8a2bf None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:17.540316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e375959-8c5c-4da6-9bad-02a139d8a2bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:17.580951 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a4bc4-24a3-488f-90d7-90878fd1e2dd None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:17.610045 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:17.630240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02220f6f-bd86-4a27-b821-e6e113546cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:17.631204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02220f6f-bd86-4a27-b821-e6e113546cb2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:17.631527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02220f6f-bd86-4a27-b821-e6e113546cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:17.631826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02220f6f-bd86-4a27-b821-e6e113546cb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:17.647219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02220f6f-bd86-4a27-b821-e6e113546cb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:17.649449 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02220f6f-bd86-4a27-b821-e6e113546cb2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:17.670503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e375959-8c5c-4da6-9bad-02a139d8a2bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-02220f6f-bd86-4a27-b821-e6e113546cb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:17.673577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:17.675896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:17.677680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:17.677680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:17.677680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:17.677680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:17.677680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 321/630] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:17.677680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02220f6f-bd86-4a27-b821-e6e113546cb2 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:17.680197 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e375959-8c5c-4da6-9bad-02a139d8a2bf None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:17.795088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:17.838736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02220f6f-bd86-4a27-b821-e6e113546cb2 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:17.854506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 310/631] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:17.855844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e375959-8c5c-4da6-9bad-02a139d8a2bf None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:17.916770 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48977f0-d424-49a6-ab3e-b15665ac1ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:17.917902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48977f0-d424-49a6-ab3e-b15665ac1ffa None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:17.918373 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48977f0-d424-49a6-ab3e-b15665ac1ffa None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:17.918666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48977f0-d424-49a6-ab3e-b15665ac1ffa None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:17.926647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48977f0-d424-49a6-ab3e-b15665ac1ffa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:17.928125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48977f0-d424-49a6-ab3e-b15665ac1ffa None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-d48977f0-d424-49a6-ab3e-b15665ac1ffa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:17.953241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:17.954422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:17.955531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:17.955531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:17.955531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:17.955531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:17.955531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:17.955531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:17.955531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:17.955531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:17.956550 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:17.958483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 322/632] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:17.960122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48977f0-d424-49a6-ab3e-b15665ac1ffa None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:17.994937 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e375959-8c5c-4da6-9bad-02a139d8a2bf None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:18.039196 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01862b17-87c7-4c26-9516-d20837210d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:18.039969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01862b17-87c7-4c26-9516-d20837210d9e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.040256 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01862b17-87c7-4c26-9516-d20837210d9e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.040548 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01862b17-87c7-4c26-9516-d20837210d9e None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.041273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01862b17-87c7-4c26-9516-d20837210d9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:18.041588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01862b17-87c7-4c26-9516-d20837210d9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:18.042989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01862b17-87c7-4c26-9516-d20837210d9e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:18.057247 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 311/633] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:18.058234 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ccf68-b3bb-462d-a07e-ab0a8b026d15 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.058577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ccf68-b3bb-462d-a07e-ab0a8b026d15 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.058979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ccf68-b3bb-462d-a07e-ab0a8b026d15 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.060475 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 312/634] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:18.061046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ccf68-b3bb-462d-a07e-ab0a8b026d15 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:18.087582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48977f0-d424-49a6-ab3e-b15665ac1ffa None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:18.118647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:18.124224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88f312-a1a7-4cf4-94ed-3e538f724bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:18.125023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88f312-a1a7-4cf4-94ed-3e538f724bc1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.125307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88f312-a1a7-4cf4-94ed-3e538f724bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.125588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88f312-a1a7-4cf4-94ed-3e538f724bc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.139117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88f312-a1a7-4cf4-94ed-3e538f724bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:18.140612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88f312-a1a7-4cf4-94ed-3e538f724bc1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-3b88f312-a1a7-4cf4-94ed-3e538f724bc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:18.160936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:18.161987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:18.163743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:18.163743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:18.163743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:18.163743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:18.163743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 323/635] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:18.164455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5978c-bffe-4bb2-9a6b-67228a0b24d3 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.165324 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5978c-bffe-4bb2-9a6b-67228a0b24d3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.165622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5978c-bffe-4bb2-9a6b-67228a0b24d3 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.167023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 324/636] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:18.167928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5978c-bffe-4bb2-9a6b-67228a0b24d3 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:18.186258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ccf68-b3bb-462d-a07e-ab0a8b026d15 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:18.217767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:18.222675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908b3d5-0106-4a3a-84ee-b5af6587615a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:18.223445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908b3d5-0106-4a3a-84ee-b5af6587615a None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.231669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908b3d5-0106-4a3a-84ee-b5af6587615a None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.231669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908b3d5-0106-4a3a-84ee-b5af6587615a None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.231669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908b3d5-0106-4a3a-84ee-b5af6587615a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:18.231669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908b3d5-0106-4a3a-84ee-b5af6587615a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:18.231669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908b3d5-0106-4a3a-84ee-b5af6587615a None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:18.240537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 313/637] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:18.242628 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52a28e-e280-42f9-aa00-517dc1d4e923 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.243048 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52a28e-e280-42f9-aa00-517dc1d4e923 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.243185 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52a28e-e280-42f9-aa00-517dc1d4e923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.263431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ef52a28e-e280-42f9-aa00-517dc1d4e923 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:18.264677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ef52a28e-e280-42f9-aa00-517dc1d4e923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:18.281806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5978c-bffe-4bb2-9a6b-67228a0b24d3 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:18.318338 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1e852-110c-4a1b-b436-dfb0d329dfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:18.319271 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1e852-110c-4a1b-b436-dfb0d329dfaf None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.319557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1e852-110c-4a1b-b436-dfb0d329dfaf None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.319836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1e852-110c-4a1b-b436-dfb0d329dfaf None admin] PATH_INFO: `/v3/users/glance` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.327749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1e852-110c-4a1b-b436-dfb0d329dfaf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:18.329227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1e852-110c-4a1b-b436-dfb0d329dfaf None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-95e1e852-110c-4a1b-b436-dfb0d329dfaf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:18.345737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:18.347291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:18.348476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:18.348476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:18.348476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:18.348476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:18.348476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:18.348476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:18.348476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:18.348476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:18.348476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 18:43:18.348476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:18.349006 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 314/638] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:18.349006 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52a28e-e280-42f9-aa00-517dc1d4e923 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:18.349148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 325/639] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:18.349148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7583da-8389-4fbb-aa84-a66a06274d24 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.349148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7583da-8389-4fbb-aa84-a66a06274d24 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.349148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7583da-8389-4fbb-aa84-a66a06274d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.371620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-0e7583da-8389-4fbb-aa84-a66a06274d24 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:18.372771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-0e7583da-8389-4fbb-aa84-a66a06274d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:18.412654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:18.449645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 326/640] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:18.450948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7583da-8389-4fbb-aa84-a66a06274d24 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:18.467911 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52a28e-e280-42f9-aa00-517dc1d4e923 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:18.496255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:18.526715 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1639816-9e87-44f4-a91c-5eb4344d8734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:18.527719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1639816-9e87-44f4-a91c-5eb4344d8734 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.528072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1639816-9e87-44f4-a91c-5eb4344d8734 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.528573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1639816-9e87-44f4-a91c-5eb4344d8734 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.529371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1639816-9e87-44f4-a91c-5eb4344d8734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:18.530040 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1639816-9e87-44f4-a91c-5eb4344d8734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:18.531516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1639816-9e87-44f4-a91c-5eb4344d8734 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:18.547028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 315/641] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:18.548038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1639816-9e87-44f4-a91c-5eb4344d8734 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:18.580177 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7583da-8389-4fbb-aa84-a66a06274d24 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:18.620880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109555b-978e-41d2-80da-2f6d19e1bcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:18.621856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109555b-978e-41d2-80da-2f6d19e1bcd1 None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.622095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109555b-978e-41d2-80da-2f6d19e1bcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.622442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109555b-978e-41d2-80da-2f6d19e1bcd1 None admin] PATH_INFO: `/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/1ee9152572224cc499229e0fa2b81ac5/roles/1f308d13b8d04c0b9309a0916dee061f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.640214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109555b-978e-41d2-80da-2f6d19e1bcd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d7c3b139f284435aea4ff08a2f0952b, user_id=1ee9152572224cc499229e0fa2b81ac5, role_id=1f308d13b8d04c0b9309a0916dee061f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:18.641688 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109555b-978e-41d2-80da-2f6d19e1bcd1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:18.661755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1639816-9e87-44f4-a91c-5eb4344d8734 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:18.699993 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211fdc1-ce0e-4b60-8eb5-35d0809476c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:18.700662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211fdc1-ce0e-4b60-8eb5-35d0809476c3 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.700944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211fdc1-ce0e-4b60-8eb5-35d0809476c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.701218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211fdc1-ce0e-4b60-8eb5-35d0809476c3 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.702057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211fdc1-ce0e-4b60-8eb5-35d0809476c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:18.702387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211fdc1-ce0e-4b60-8eb5-35d0809476c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:18.703810 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211fdc1-ce0e-4b60-8eb5-35d0809476c3 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:18.715008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 327/642] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:18 2022] PUT /identity/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/1ee9152572224cc499229e0fa2b81ac5/roles/1f308d13b8d04c0b9309a0916dee061f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:18.716089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109555b-978e-41d2-80da-2f6d19e1bcd1 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:18.729406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 316/643] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Jul 25 18:43:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:18.730360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211fdc1-ce0e-4b60-8eb5-35d0809476c3 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:18.750169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:18.787187 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:18.855738 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109555b-978e-41d2-80da-2f6d19e1bcd1 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:18.858643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211fdc1-ce0e-4b60-8eb5-35d0809476c3 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:18.911361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebd015-0491-41d4-8947-2c690e291c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:18.912662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebd015-0491-41d4-8947-2c690e291c00 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.913859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebd015-0491-41d4-8947-2c690e291c00 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.914364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebd015-0491-41d4-8947-2c690e291c00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.917613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbee51-0e57-4b5c-8469-d280514073a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:18.919046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbee51-0e57-4b5c-8469-d280514073a3 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:18.919554 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbee51-0e57-4b5c-8469-d280514073a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:18.920036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbee51-0e57-4b5c-8469-d280514073a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:18.929634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ebd015-0491-41d4-8947-2c690e291c00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:18.936968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbee51-0e57-4b5c-8469-d280514073a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-40ebd015-0491-41d4-8947-2c690e291c00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:18.938651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:18.939618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:18.940468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:18.940468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:18.940468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:18.940468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:18.940468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:18.940468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:18.940468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:18.940468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:18.940468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:18.940468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:18.940468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:18.940468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:18.942825 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 328/644] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:18.943790 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebd015-0491-41d4-8947-2c690e291c00 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-23dbee51-0e57-4b5c-8469-d280514073a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:18.945073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:18.946171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:18.947099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:18.947099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:18.947099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:18.947099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:18.947099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:18.947099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:18.947099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:18.947099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:18.947099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:18.947099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:18.947099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:18.947099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:18.947099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 317/645] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:18.948025 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbee51-0e57-4b5c-8469-d280514073a3 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.085080 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:19.090735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbee51-0e57-4b5c-8469-d280514073a3 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.095352 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:19.119789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebd015-0491-41d4-8947-2c690e291c00 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.139142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b776f2-6642-445d-ab95-dacfcea0189e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:19.139835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b776f2-6642-445d-ab95-dacfcea0189e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.140150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b776f2-6642-445d-ab95-dacfcea0189e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.140409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b776f2-6642-445d-ab95-dacfcea0189e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.170253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37beae47-221c-42b7-bb19-d68e2a884782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:19.171077 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37beae47-221c-42b7-bb19-d68e2a884782 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.171368 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37beae47-221c-42b7-bb19-d68e2a884782 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.171802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37beae47-221c-42b7-bb19-d68e2a884782 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.180423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37beae47-221c-42b7-bb19-d68e2a884782 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:19.182009 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37beae47-221c-42b7-bb19-d68e2a884782 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-37beae47-221c-42b7-bb19-d68e2a884782 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:19.198610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:19.199676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:19.200671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:19.200671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:19.200671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:19.200671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:19.200671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:19.200671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:19.200671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:19.200671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:19.200671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 329/646] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:19.201151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37beae47-221c-42b7-bb19-d68e2a884782 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.277362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b776f2-6642-445d-ab95-dacfcea0189e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:19.278986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b776f2-6642-445d-ab95-dacfcea0189e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:19.327057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37beae47-221c-42b7-bb19-d68e2a884782 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.368033 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2143d-c9a6-405d-9af2-9f3d04da2aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:19.368929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2143d-c9a6-405d-9af2-9f3d04da2aa3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.369228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2143d-c9a6-405d-9af2-9f3d04da2aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.369512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2143d-c9a6-405d-9af2-9f3d04da2aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.370570 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2143d-c9a6-405d-9af2-9f3d04da2aa3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:19.370891 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2143d-c9a6-405d-9af2-9f3d04da2aa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:19.372305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2143d-c9a6-405d-9af2-9f3d04da2aa3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:19.381560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 330/647] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:19.383156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2143d-c9a6-405d-9af2-9f3d04da2aa3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.389138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:19.412127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:19.417554 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 318/648] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:19.418694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b776f2-6642-445d-ab95-dacfcea0189e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.516908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2143d-c9a6-405d-9af2-9f3d04da2aa3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.555443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b776f2-6642-445d-ab95-dacfcea0189e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.558536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976ee97-8055-45dd-9776-fccabfc8f8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:19.559246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976ee97-8055-45dd-9776-fccabfc8f8e1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.559572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976ee97-8055-45dd-9776-fccabfc8f8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.559855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976ee97-8055-45dd-9776-fccabfc8f8e1 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.560605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976ee97-8055-45dd-9776-fccabfc8f8e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:19.560911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976ee97-8055-45dd-9776-fccabfc8f8e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:19.562410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976ee97-8055-45dd-9776-fccabfc8f8e1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:19.586144 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 331/649] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:19.586144 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976ee97-8055-45dd-9776-fccabfc8f8e1 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.615507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670786c-4f70-490e-8e83-f018b5ceaec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:19.617239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670786c-4f70-490e-8e83-f018b5ceaec2 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.617537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670786c-4f70-490e-8e83-f018b5ceaec2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.617811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670786c-4f70-490e-8e83-f018b5ceaec2 None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.618774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670786c-4f70-490e-8e83-f018b5ceaec2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:19.619190 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670786c-4f70-490e-8e83-f018b5ceaec2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:19.620965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670786c-4f70-490e-8e83-f018b5ceaec2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:19.642346 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 319/650] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:19.644041 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670786c-4f70-490e-8e83-f018b5ceaec2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.684626 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:19.727009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:19.729767 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976ee97-8055-45dd-9776-fccabfc8f8e1 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.772255 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7043249-8b05-47cd-86ba-6c39c7d7836c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:19.774375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7043249-8b05-47cd-86ba-6c39c7d7836c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.774689 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7043249-8b05-47cd-86ba-6c39c7d7836c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.774965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7043249-8b05-47cd-86ba-6c39c7d7836c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.784784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670786c-4f70-490e-8e83-f018b5ceaec2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.787166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7043249-8b05-47cd-86ba-6c39c7d7836c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:19.788713 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7043249-8b05-47cd-86ba-6c39c7d7836c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-b7043249-8b05-47cd-86ba-6c39c7d7836c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:19.813574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:19.814820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:19.815813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:19.815813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:19.815813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:19.815813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:19.815813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 332/651] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:19.816512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7043249-8b05-47cd-86ba-6c39c7d7836c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.845624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ac48f-cf9c-47fb-afb3-d75badbfbc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:19.846579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ac48f-cf9c-47fb-afb3-d75badbfbc81 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:19.846945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ac48f-cf9c-47fb-afb3-d75badbfbc81 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:19.847416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ac48f-cf9c-47fb-afb3-d75badbfbc81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:19.864158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ac48f-cf9c-47fb-afb3-d75badbfbc81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:19.865618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ac48f-cf9c-47fb-afb3-d75badbfbc81 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-917ac48f-cf9c-47fb-afb3-d75badbfbc81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:19.886655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:19.887900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:19.888800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:19.888800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:19.888800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:19.888800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:19.888800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 320/652] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:19.888800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ac48f-cf9c-47fb-afb3-d75badbfbc81 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:19.960058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7043249-8b05-47cd-86ba-6c39c7d7836c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:19.981615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:20.004802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34f785-d370-423f-ac80-9babacba2f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:20.005625 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34f785-d370-423f-ac80-9babacba2f51 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:20.006041 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34f785-d370-423f-ac80-9babacba2f51 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:20.006221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34f785-d370-423f-ac80-9babacba2f51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:20.020203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34f785-d370-423f-ac80-9babacba2f51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:20.021678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34f785-d370-423f-ac80-9babacba2f51 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:20.022214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ac48f-cf9c-47fb-afb3-d75badbfbc81 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:20.042925 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-fe34f785-d370-423f-ac80-9babacba2f51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:20.044315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:20.045809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:20.047057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:20.047057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:20.047057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:20.047057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:20.047057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 333/653] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:20.047057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34f785-d370-423f-ac80-9babacba2f51 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:20.079632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e3d33-71b6-4ef4-8a26-423545eecec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:20.080704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e3d33-71b6-4ef4-8a26-423545eecec6 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:20.081200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e3d33-71b6-4ef4-8a26-423545eecec6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:20.082390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e3d33-71b6-4ef4-8a26-423545eecec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:20.083535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e3d33-71b6-4ef4-8a26-423545eecec6 None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=e628cb75e0c74f27a390945c8a04335c) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:20.084279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e3d33-71b6-4ef4-8a26-423545eecec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:20.086920 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e3d33-71b6-4ef4-8a26-423545eecec6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:20.096409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 321/654] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:19 2022] GET /identity/v3/role_assignments?user.id=e628cb75e0c74f27a390945c8a04335c&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:20.097664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e3d33-71b6-4ef4-8a26-423545eecec6 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:20.245051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34f785-d370-423f-ac80-9babacba2f51 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:20.308450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e3d33-71b6-4ef4-8a26-423545eecec6 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:20.339732 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fc976-459c-4273-90d3-42a735274684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:20.340865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fc976-459c-4273-90d3-42a735274684 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:20.341355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fc976-459c-4273-90d3-42a735274684 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:20.341848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fc976-459c-4273-90d3-42a735274684 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:20.343372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fc976-459c-4273-90d3-42a735274684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:20.343652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fc976-459c-4273-90d3-42a735274684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:20.346115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fc976-459c-4273-90d3-42a735274684 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:20.356137 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac685ab-6207-4941-b4c6-12b203726612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:20.356681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac685ab-6207-4941-b4c6-12b203726612 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:20.357032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac685ab-6207-4941-b4c6-12b203726612 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:20.357349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac685ab-6207-4941-b4c6-12b203726612 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:20.358199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac685ab-6207-4941-b4c6-12b203726612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:20.358543 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac685ab-6207-4941-b4c6-12b203726612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:20.360481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac685ab-6207-4941-b4c6-12b203726612 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:20.370484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 334/655] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:20.371388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fc976-459c-4273-90d3-42a735274684 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:20.381307 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 322/656] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:20.387648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac685ab-6207-4941-b4c6-12b203726612 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:20.425221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:20.453540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:20.562604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fc976-459c-4273-90d3-42a735274684 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:20.598036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac685ab-6207-4941-b4c6-12b203726612 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:20.606909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c70907-a7b5-4f11-aba2-11b3533b7fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:20.607769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c70907-a7b5-4f11-aba2-11b3533b7fed None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:20.608124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c70907-a7b5-4f11-aba2-11b3533b7fed None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:20.608478 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c70907-a7b5-4f11-aba2-11b3533b7fed None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:20.609363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c70907-a7b5-4f11-aba2-11b3533b7fed None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:20.609771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c70907-a7b5-4f11-aba2-11b3533b7fed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:20.612106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c70907-a7b5-4f11-aba2-11b3533b7fed None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:20.631519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 335/657] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:20.632652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c70907-a7b5-4f11-aba2-11b3533b7fed None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:20.649274 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4eb9c5-e724-4177-b112-461209b92854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:20.650636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4eb9c5-e724-4177-b112-461209b92854 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:20.651074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4eb9c5-e724-4177-b112-461209b92854 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:20.652017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4eb9c5-e724-4177-b112-461209b92854 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:20.662736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4eb9c5-e724-4177-b112-461209b92854 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:20.665181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4eb9c5-e724-4177-b112-461209b92854 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-ba4eb9c5-e724-4177-b112-461209b92854 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:20.682643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:20.683644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:20.684569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:20.684569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:20.684569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:20.684569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:20.684569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:20.684569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:20.684569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:20.684569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:20.684569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:20.684569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:20.689499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 323/658] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:20.690656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4eb9c5-e724-4177-b112-461209b92854 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:20.756460 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:20.772358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c70907-a7b5-4f11-aba2-11b3533b7fed None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:20.800478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:20.809559 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4eb9c5-e724-4177-b112-461209b92854 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:20.827507 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10d247-654e-49b4-857f-5f829083f7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:20.829068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10d247-654e-49b4-857f-5f829083f7a1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:20.829649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10d247-654e-49b4-857f-5f829083f7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:20.830104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10d247-654e-49b4-857f-5f829083f7a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:20.839831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10d247-654e-49b4-857f-5f829083f7a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:20.842212 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10d247-654e-49b4-857f-5f829083f7a1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:20.852862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211bad4-53f4-431b-b9b9-2a3d1ea1d7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:20.853821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211bad4-53f4-431b-b9b9-2a3d1ea1d7bb None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:20.854211 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211bad4-53f4-431b-b9b9-2a3d1ea1d7bb None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:20.854616 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211bad4-53f4-431b-b9b9-2a3d1ea1d7bb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-4c10d247-654e-49b4-857f-5f829083f7a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:20.861520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:20.862500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:20.863386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:20.863386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:20.863386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:20.863386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:20.863386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:20.863386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:20.863386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:20.863386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:20.863386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 336/659] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:20.864700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10d247-654e-49b4-857f-5f829083f7a1 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:20.866947 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211bad4-53f4-431b-b9b9-2a3d1ea1d7bb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:20.868963 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211bad4-53f4-431b-b9b9-2a3d1ea1d7bb None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-1211bad4-53f4-431b-b9b9-2a3d1ea1d7bb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:20.887493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:20.888498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:20.889418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:20.889418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:20.889418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:20.889418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:20.889418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:20.889418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:20.889418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:20.889418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:20.889418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 18:43:20.889418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:20.893593 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 324/660] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:20.894631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211bad4-53f4-431b-b9b9-2a3d1ea1d7bb None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:21.009849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10d247-654e-49b4-857f-5f829083f7a1 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:21.011075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211bad4-53f4-431b-b9b9-2a3d1ea1d7bb None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:21.057466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dce7df-6150-41e5-b9b4-af6bfda9194d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:21.058566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dce7df-6150-41e5-b9b4-af6bfda9194d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:21.059048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dce7df-6150-41e5-b9b4-af6bfda9194d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:21.059487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dce7df-6150-41e5-b9b4-af6bfda9194d None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:21.060552 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dce7df-6150-41e5-b9b4-af6bfda9194d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:21.061062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dce7df-6150-41e5-b9b4-af6bfda9194d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:21.062923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dce7df-6150-41e5-b9b4-af6bfda9194d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:21.066445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec4f7d-5c77-44a0-8528-6a1afff145fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:21.067165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec4f7d-5c77-44a0-8528-6a1afff145fe None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:21.067450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec4f7d-5c77-44a0-8528-6a1afff145fe None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:21.067736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec4f7d-5c77-44a0-8528-6a1afff145fe None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:21.068561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec4f7d-5c77-44a0-8528-6a1afff145fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:21.068875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec4f7d-5c77-44a0-8528-6a1afff145fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:21.070928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec4f7d-5c77-44a0-8528-6a1afff145fe None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:21.084827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 325/661] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:21.085997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec4f7d-5c77-44a0-8528-6a1afff145fe None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:21.094927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 337/662] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Jul 25 18:43:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:21.096669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5b001-e6c3-4e77-8a42-1e80a230f079 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:21.096830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5b001-e6c3-4e77-8a42-1e80a230f079 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:21.097272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5b001-e6c3-4e77-8a42-1e80a230f079 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:21.103769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 338/663] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:21.105169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5b001-e6c3-4e77-8a42-1e80a230f079 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:21.119397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:21.126159 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:21.225090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec4f7d-5c77-44a0-8528-6a1afff145fe None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:21.243621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5b001-e6c3-4e77-8a42-1e80a230f079 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:21.264089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0c045-8d3b-4c79-81fd-784c6f5854c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:21.264901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0c045-8d3b-4c79-81fd-784c6f5854c1 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:21.265827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0c045-8d3b-4c79-81fd-784c6f5854c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:21.265827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0c045-8d3b-4c79-81fd-784c6f5854c1 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:21.266630 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0c045-8d3b-4c79-81fd-784c6f5854c1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:21.266942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0c045-8d3b-4c79-81fd-784c6f5854c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:21.268455 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0c045-8d3b-4c79-81fd-784c6f5854c1 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:21.289334 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 326/664] 10.222.0.17 () {56 vars in 1139 bytes} [Mon Jul 25 18:43:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:21.290629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0c045-8d3b-4c79-81fd-784c6f5854c1 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:21.297838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ebe8b-4066-47ed-9e04-55ea55d4c977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:21.299877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ebe8b-4066-47ed-9e04-55ea55d4c977 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:21.304319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ebe8b-4066-47ed-9e04-55ea55d4c977 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:21.304319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ebe8b-4066-47ed-9e04-55ea55d4c977 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:21.304319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ebe8b-4066-47ed-9e04-55ea55d4c977 None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=28954d67689a46ee975fdfc9139e41eb) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:21.304319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ebe8b-4066-47ed-9e04-55ea55d4c977 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:21.306619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ebe8b-4066-47ed-9e04-55ea55d4c977 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:21.318780 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 339/665] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:21 2022] GET /identity/v3/role_assignments?user.id=28954d67689a46ee975fdfc9139e41eb&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:21.321614 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9620a-9f50-44d4-8bb3-baf8f4b547df None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:21.322074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9620a-9f50-44d4-8bb3-baf8f4b547df None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:21.322582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9620a-9f50-44d4-8bb3-baf8f4b547df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:21.350638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-aff9620a-9f50-44d4-8bb3-baf8f4b547df None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:21.353341 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-aff9620a-9f50-44d4-8bb3-baf8f4b547df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:21.434491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:21.462771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 340/666] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:21.463846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9620a-9f50-44d4-8bb3-baf8f4b547df None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:21.467144 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:21.475933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0c045-8d3b-4c79-81fd-784c6f5854c1 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:21.528348 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e6a29-5812-40be-a14c-f15e0e2710ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:21.529456 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e6a29-5812-40be-a14c-f15e0e2710ba None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:21.529946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e6a29-5812-40be-a14c-f15e0e2710ba None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:21.530961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e6a29-5812-40be-a14c-f15e0e2710ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:21.621996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e6a29-5812-40be-a14c-f15e0e2710ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:21.623103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9620a-9f50-44d4-8bb3-baf8f4b547df None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:21.624977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e6a29-5812-40be-a14c-f15e0e2710ba None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:21.675895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e188ad7-f920-47c6-b4fd-471bf364acc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:21.676727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e188ad7-f920-47c6-b4fd-471bf364acc2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:21.677020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e188ad7-f920-47c6-b4fd-471bf364acc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:21.677484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e188ad7-f920-47c6-b4fd-471bf364acc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:21.785134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e188ad7-f920-47c6-b4fd-471bf364acc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:21.787441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e188ad7-f920-47c6-b4fd-471bf364acc2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:21.790798 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:21.854395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:21.890949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 327/667] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:21.891880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e6a29-5812-40be-a14c-f15e0e2710ba None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:21.919789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 341/668] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:21.920876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e188ad7-f920-47c6-b4fd-471bf364acc2 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.039901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e188ad7-f920-47c6-b4fd-471bf364acc2 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.041355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e6a29-5812-40be-a14c-f15e0e2710ba None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.042104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:22.079754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f5dd7-8a36-4799-b940-9315a0a32cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.081128 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f5dd7-8a36-4799-b940-9315a0a32cdf None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.081716 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f5dd7-8a36-4799-b940-9315a0a32cdf None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.082291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f5dd7-8a36-4799-b940-9315a0a32cdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.084064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792eba1-be8b-4886-bb38-3c65badb5619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.084876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792eba1-be8b-4886-bb38-3c65badb5619 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.085382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792eba1-be8b-4886-bb38-3c65badb5619 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.085680 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792eba1-be8b-4886-bb38-3c65badb5619 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.095420 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f5dd7-8a36-4799-b940-9315a0a32cdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:22.097365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f5dd7-8a36-4799-b940-9315a0a32cdf None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:22.098549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792eba1-be8b-4886-bb38-3c65badb5619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-1792eba1-be8b-4886-bb38-3c65badb5619 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:22.108835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:22.109839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:22.110720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:22.110720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:22.110720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:22.110720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:22.110720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:22.110720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:22.110720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:22.110720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:22.110720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:22.110720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:22.110720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:22.110720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:22.114710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 328/669] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:22.115591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792eba1-be8b-4886-bb38-3c65badb5619 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-098f5dd7-8a36-4799-b940-9315a0a32cdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:22.121148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:22.122029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:22.122944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:22.122944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:22.122944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:22.122944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:22.125147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 342/670] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:22.126280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f5dd7-8a36-4799-b940-9315a0a32cdf None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.166729 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:22.263696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f5dd7-8a36-4799-b940-9315a0a32cdf None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.264384 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792eba1-be8b-4886-bb38-3c65badb5619 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.307997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9ba8f-8c98-4de8-8335-8148c4f34a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.309204 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9ba8f-8c98-4de8-8335-8148c4f34a11 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.309741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9ba8f-8c98-4de8-8335-8148c4f34a11 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.310411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9ba8f-8c98-4de8-8335-8148c4f34a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.314023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6ca68-3a88-40c2-9a48-ca1e02b3d1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.315182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6ca68-3a88-40c2-9a48-ca1e02b3d1e7 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.315476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6ca68-3a88-40c2-9a48-ca1e02b3d1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.315761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6ca68-3a88-40c2-9a48-ca1e02b3d1e7 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.316533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6ca68-3a88-40c2-9a48-ca1e02b3d1e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:22.316851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6ca68-3a88-40c2-9a48-ca1e02b3d1e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:22.318256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6ca68-3a88-40c2-9a48-ca1e02b3d1e7 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:22.325128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9ba8f-8c98-4de8-8335-8148c4f34a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:22.326965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9ba8f-8c98-4de8-8335-8148c4f34a11 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:22.330547 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 343/671] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:22.334506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6ca68-3a88-40c2-9a48-ca1e02b3d1e7 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-ace9ba8f-8c98-4de8-8335-8148c4f34a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:22.354109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:22.355203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:22.356585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:22.356585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:22.356585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:22.356585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:22.356585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 329/672] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:22.359149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9ba8f-8c98-4de8-8335-8148c4f34a11 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.366850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:22.465316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:22.473652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6ca68-3a88-40c2-9a48-ca1e02b3d1e7 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.484917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9ba8f-8c98-4de8-8335-8148c4f34a11 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.513262 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a9abe-d0d3-45fd-a760-cec181beecd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.514069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a9abe-d0d3-45fd-a760-cec181beecd6 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.514671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a9abe-d0d3-45fd-a760-cec181beecd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.514671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a9abe-d0d3-45fd-a760-cec181beecd6 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.515385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a9abe-d0d3-45fd-a760-cec181beecd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:22.515699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a9abe-d0d3-45fd-a760-cec181beecd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:22.517382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a9abe-d0d3-45fd-a760-cec181beecd6 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:22.525832 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541e392-5452-4688-bc33-73f58cd68062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.526678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541e392-5452-4688-bc33-73f58cd68062 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.526964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541e392-5452-4688-bc33-73f58cd68062 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.527243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541e392-5452-4688-bc33-73f58cd68062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.530858 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 344/673] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:22.531800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a9abe-d0d3-45fd-a760-cec181beecd6 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.541996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541e392-5452-4688-bc33-73f58cd68062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:22.543993 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541e392-5452-4688-bc33-73f58cd68062 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-6541e392-5452-4688-bc33-73f58cd68062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:22.575383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:22.576491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:22.577454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:22.577454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:22.577454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:22.577454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:22.583248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 330/674] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:22.584769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88383302-71fb-4000-9e2d-02b167b9a459 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.585291 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88383302-71fb-4000-9e2d-02b167b9a459 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.585711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88383302-71fb-4000-9e2d-02b167b9a459 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.590374 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 331/675] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:22.590374 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88383302-71fb-4000-9e2d-02b167b9a459 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.633517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:22.656597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a9abe-d0d3-45fd-a760-cec181beecd6 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.695464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f730e5-1960-42de-98c9-63df179e9367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.696204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f730e5-1960-42de-98c9-63df179e9367 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.696488 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f730e5-1960-42de-98c9-63df179e9367 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.696766 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f730e5-1960-42de-98c9-63df179e9367 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.697668 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f730e5-1960-42de-98c9-63df179e9367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:22.697980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f730e5-1960-42de-98c9-63df179e9367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:22.699413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f730e5-1960-42de-98c9-63df179e9367 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:22.718228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 345/676] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:22.718228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f730e5-1960-42de-98c9-63df179e9367 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.726897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88383302-71fb-4000-9e2d-02b167b9a459 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.783127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe67c9-09d0-4df0-a674-401bf33594ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.783939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe67c9-09d0-4df0-a674-401bf33594ae None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.784308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe67c9-09d0-4df0-a674-401bf33594ae None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.784584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe67c9-09d0-4df0-a674-401bf33594ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.793424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe67c9-09d0-4df0-a674-401bf33594ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:22.795016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe67c9-09d0-4df0-a674-401bf33594ae None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:22.797166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-70fe67c9-09d0-4df0-a674-401bf33594ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:22.814977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:22.816808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:22.818645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:22.818645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:22.818645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:22.818645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:22.818645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:22.818645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:22.818645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:22.818645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:22.822713 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 332/677] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:22.823995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee8db-29b1-4a78-9de2-e9650c6ab1ed None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.824380 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee8db-29b1-4a78-9de2-e9650c6ab1ed None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.824671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee8db-29b1-4a78-9de2-e9650c6ab1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.843585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-557ee8db-29b1-4a78-9de2-e9650c6ab1ed None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:22.844802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-557ee8db-29b1-4a78-9de2-e9650c6ab1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:22.871550 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f730e5-1960-42de-98c9-63df179e9367 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:22.916003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f9ff6-219c-4d7c-81fd-d9b4eddae68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:22.916774 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f9ff6-219c-4d7c-81fd-d9b4eddae68f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:22.917067 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f9ff6-219c-4d7c-81fd-d9b4eddae68f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:22.917351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f9ff6-219c-4d7c-81fd-d9b4eddae68f None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:22.918124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f9ff6-219c-4d7c-81fd-d9b4eddae68f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:22.918480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f9ff6-219c-4d7c-81fd-d9b4eddae68f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:22.919878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f9ff6-219c-4d7c-81fd-d9b4eddae68f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:22.924214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:22.933321 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 333/678] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:22.934237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ee8db-29b1-4a78-9de2-e9650c6ab1ed None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:22.941940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 346/679] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:22.943765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f9ff6-219c-4d7c-81fd-d9b4eddae68f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.047619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ee8db-29b1-4a78-9de2-e9650c6ab1ed None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.056977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f9ff6-219c-4d7c-81fd-d9b4eddae68f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.083974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.092885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e7ba4-3b84-4a68-8f1b-d5b2824fe6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.093696 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e7ba4-3b84-4a68-8f1b-d5b2824fe6db None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.093985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e7ba4-3b84-4a68-8f1b-d5b2824fe6db None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.094271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e7ba4-3b84-4a68-8f1b-d5b2824fe6db None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.094619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2498f-b49d-4687-96ab-40f1f0f6fb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.100465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2498f-b49d-4687-96ab-40f1f0f6fb51 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.100465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2498f-b49d-4687-96ab-40f1f0f6fb51 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.100465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2498f-b49d-4687-96ab-40f1f0f6fb51 None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.100465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2498f-b49d-4687-96ab-40f1f0f6fb51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:23.100465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2498f-b49d-4687-96ab-40f1f0f6fb51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.100465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2498f-b49d-4687-96ab-40f1f0f6fb51 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.102152 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e7ba4-3b84-4a68-8f1b-d5b2824fe6db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.103639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e7ba4-3b84-4a68-8f1b-d5b2824fe6db None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.114658 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 347/680] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:23.115862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2498f-b49d-4687-96ab-40f1f0f6fb51 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-126e7ba4-3b84-4a68-8f1b-d5b2824fe6db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:23.118622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:23.119824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:23.120718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:23.120718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:23.120718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:23.120718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:23.120718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:23.120718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:23.120718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:23.120718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:23.124634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 334/681] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:23.125345 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e7ba4-3b84-4a68-8f1b-d5b2824fe6db None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.190389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.237714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2498f-b49d-4687-96ab-40f1f0f6fb51 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.248300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e7ba4-3b84-4a68-8f1b-d5b2824fe6db None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.279025 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7e1ee-e6f0-405e-ba1d-8f26597b7dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.279844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7e1ee-e6f0-405e-ba1d-8f26597b7dd5 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.280130 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7e1ee-e6f0-405e-ba1d-8f26597b7dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.280410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7e1ee-e6f0-405e-ba1d-8f26597b7dd5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.285652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb01a2e-438b-44aa-80b4-cbbe7e32eda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.286485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb01a2e-438b-44aa-80b4-cbbe7e32eda8 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.286776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb01a2e-438b-44aa-80b4-cbbe7e32eda8 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.287056 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb01a2e-438b-44aa-80b4-cbbe7e32eda8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.288776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7e1ee-e6f0-405e-ba1d-8f26597b7dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.290373 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7e1ee-e6f0-405e-ba1d-8f26597b7dd5 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.299512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb01a2e-438b-44aa-80b4-cbbe7e32eda8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-10f7e1ee-e6f0-405e-ba1d-8f26597b7dd5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:23.305143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:23.306802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:23.307754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:23.307754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:23.307754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:23.307754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:23.307754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:23.307754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:23.307754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:23.307754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:23.307754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:23.307754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:23.307754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 348/682] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:23.307754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7e1ee-e6f0-405e-ba1d-8f26597b7dd5 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-ebb01a2e-438b-44aa-80b4-cbbe7e32eda8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:23.309880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:23.310974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:23.311786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:23.311786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:23.311786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:23.311786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:23.311786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:23.311786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:23.311786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:23.311786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:23.311786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:23.311786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:23.311786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:23.311786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:23.315162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 335/683] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:23.316011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb01a2e-438b-44aa-80b4-cbbe7e32eda8 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.376327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.429760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7e1ee-e6f0-405e-ba1d-8f26597b7dd5 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.437748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb01a2e-438b-44aa-80b4-cbbe7e32eda8 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.448650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.467883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba3a34-9988-44ae-aaae-363009970f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.468681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba3a34-9988-44ae-aaae-363009970f18 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.469042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba3a34-9988-44ae-aaae-363009970f18 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.469336 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba3a34-9988-44ae-aaae-363009970f18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.470195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba3a34-9988-44ae-aaae-363009970f18 None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=9a419d02a7dc46e9b3c5b3d4eb3151d6) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:23.470543 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba3a34-9988-44ae-aaae-363009970f18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.472026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba3a34-9988-44ae-aaae-363009970f18 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.477676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f735590-0fe3-493d-9197-19a02248f6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.478374 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f735590-0fe3-493d-9197-19a02248f6f0 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.478667 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f735590-0fe3-493d-9197-19a02248f6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.478965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f735590-0fe3-493d-9197-19a02248f6f0 None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.490358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 349/684] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/role_assignments?user.id=9a419d02a7dc46e9b3c5b3d4eb3151d6&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:23.490625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f735590-0fe3-493d-9197-19a02248f6f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:23.490625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f735590-0fe3-493d-9197-19a02248f6f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.490807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba3a34-9988-44ae-aaae-363009970f18 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.490891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f735590-0fe3-493d-9197-19a02248f6f0 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.506402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 336/685] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:23.507840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f735590-0fe3-493d-9197-19a02248f6f0 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.617114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba3a34-9988-44ae-aaae-363009970f18 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.651067 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f735590-0fe3-493d-9197-19a02248f6f0 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.656427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35527c-69fe-49d1-8d1b-57b61b28f8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.657126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35527c-69fe-49d1-8d1b-57b61b28f8d3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.657412 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35527c-69fe-49d1-8d1b-57b61b28f8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.657692 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35527c-69fe-49d1-8d1b-57b61b28f8d3 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.658645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35527c-69fe-49d1-8d1b-57b61b28f8d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:23.658966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35527c-69fe-49d1-8d1b-57b61b28f8d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.660483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35527c-69fe-49d1-8d1b-57b61b28f8d3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.663212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.687209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 350/686] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:23.688553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35527c-69fe-49d1-8d1b-57b61b28f8d3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.691799 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa81a0b-90cc-477a-af2b-e91ca87ba2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.692477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa81a0b-90cc-477a-af2b-e91ca87ba2a7 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.692801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa81a0b-90cc-477a-af2b-e91ca87ba2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.693089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa81a0b-90cc-477a-af2b-e91ca87ba2a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.697495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa81a0b-90cc-477a-af2b-e91ca87ba2a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:23.697495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa81a0b-90cc-477a-af2b-e91ca87ba2a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.697495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa81a0b-90cc-477a-af2b-e91ca87ba2a7 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.707536 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 337/687] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:23.708880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905644a2-4aa4-4881-b99a-22b6c88bb423 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.709175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905644a2-4aa4-4881-b99a-22b6c88bb423 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.709454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905644a2-4aa4-4881-b99a-22b6c88bb423 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.713693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 338/688] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:23.713693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905644a2-4aa4-4881-b99a-22b6c88bb423 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.746406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:23.814278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35527c-69fe-49d1-8d1b-57b61b28f8d3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.837179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905644a2-4aa4-4881-b99a-22b6c88bb423 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:23.856235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dcd3d-dbe9-4148-8689-e31f860ede0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.857026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dcd3d-dbe9-4148-8689-e31f860ede0d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.857182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dcd3d-dbe9-4148-8689-e31f860ede0d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.857470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dcd3d-dbe9-4148-8689-e31f860ede0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.858299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dcd3d-dbe9-4148-8689-e31f860ede0d None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=ffcf280bc6d54b2fb2d22634468ac6cd) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:23.858651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dcd3d-dbe9-4148-8689-e31f860ede0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:23.860181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dcd3d-dbe9-4148-8689-e31f860ede0d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:23.866779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 351/689] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/role_assignments?user.id=ffcf280bc6d54b2fb2d22634468ac6cd&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:23.867471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dcd3d-dbe9-4148-8689-e31f860ede0d None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:23.891383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625353e-0ea8-4c34-a04b-22eb059b6e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:23.892718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625353e-0ea8-4c34-a04b-22eb059b6e10 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:23.893681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625353e-0ea8-4c34-a04b-22eb059b6e10 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:23.894149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625353e-0ea8-4c34-a04b-22eb059b6e10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:23.956717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.008648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dcd3d-dbe9-4148-8689-e31f860ede0d None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.012273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.015387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625353e-0ea8-4c34-a04b-22eb059b6e10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.017069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625353e-0ea8-4c34-a04b-22eb059b6e10 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.072787 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdf648-356d-4af0-b3f2-49ace3372b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.074222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf648-356d-4af0-b3f2-49ace3372b55 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.074719 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf648-356d-4af0-b3f2-49ace3372b55 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.075181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf648-356d-4af0-b3f2-49ace3372b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.095944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdf648-356d-4af0-b3f2-49ace3372b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.098823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdf648-356d-4af0-b3f2-49ace3372b55 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-6bfdf648-356d-4af0-b3f2-49ace3372b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:24.134466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:24.135369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:24.136256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:24.136256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:24.136256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:24.136256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:24.136754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 352/690] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:24.139026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c239be-1a76-4f61-97f4-4b58a92117c9 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.139452 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c239be-1a76-4f61-97f4-4b58a92117c9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.139746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c239be-1a76-4f61-97f4-4b58a92117c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.154206 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 339/691] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:24.155280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625353e-0ea8-4c34-a04b-22eb059b6e10 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.169472 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-60c239be-1a76-4f61-97f4-4b58a92117c9 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:24.170933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-60c239be-1a76-4f61-97f4-4b58a92117c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:24.253711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.279094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 353/692] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:24.280793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c239be-1a76-4f61-97f4-4b58a92117c9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.297701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625353e-0ea8-4c34-a04b-22eb059b6e10 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.353289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf78b9-ffad-402f-be16-3bb0af0d6576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.354436 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf78b9-ffad-402f-be16-3bb0af0d6576 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.354934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf78b9-ffad-402f-be16-3bb0af0d6576 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.355410 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf78b9-ffad-402f-be16-3bb0af0d6576 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.356397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf78b9-ffad-402f-be16-3bb0af0d6576 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:24.356990 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf78b9-ffad-402f-be16-3bb0af0d6576 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.358631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf78b9-ffad-402f-be16-3bb0af0d6576 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.373624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 340/693] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:24.374934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf78b9-ffad-402f-be16-3bb0af0d6576 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.407555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.433383 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c239be-1a76-4f61-97f4-4b58a92117c9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.480303 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce83a04-25e8-486d-adf4-1529e3c5e9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.481471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce83a04-25e8-486d-adf4-1529e3c5e9c3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.481958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce83a04-25e8-486d-adf4-1529e3c5e9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.482480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce83a04-25e8-486d-adf4-1529e3c5e9c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.495170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce83a04-25e8-486d-adf4-1529e3c5e9c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.497504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce83a04-25e8-486d-adf4-1529e3c5e9c3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.502270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf78b9-ffad-402f-be16-3bb0af0d6576 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-2ce83a04-25e8-486d-adf4-1529e3c5e9c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:24.518568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:24.519505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:24.520372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:24.520372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:24.520372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:24.520372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:24.523337 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 354/694] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:24.524555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce83a04-25e8-486d-adf4-1529e3c5e9c3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.535977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.543904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822b864-cf31-4e64-a996-8dac3596c19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.544977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822b864-cf31-4e64-a996-8dac3596c19d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.545262 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822b864-cf31-4e64-a996-8dac3596c19d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.545539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822b864-cf31-4e64-a996-8dac3596c19d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.559140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822b864-cf31-4e64-a996-8dac3596c19d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.561091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822b864-cf31-4e64-a996-8dac3596c19d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-b822b864-cf31-4e64-a996-8dac3596c19d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:24.585914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:24.586910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:24.587791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:24.587791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:24.587791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:24.587791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:24.587791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 341/695] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:24.588513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822b864-cf31-4e64-a996-8dac3596c19d None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.661781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce83a04-25e8-486d-adf4-1529e3c5e9c3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.719993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ad89a-13c5-4bc2-8ee5-acd38ef96bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.720821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ad89a-13c5-4bc2-8ee5-acd38ef96bf9 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.721109 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ad89a-13c5-4bc2-8ee5-acd38ef96bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.721387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ad89a-13c5-4bc2-8ee5-acd38ef96bf9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.725707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.736987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ad89a-13c5-4bc2-8ee5-acd38ef96bf9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.739705 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ad89a-13c5-4bc2-8ee5-acd38ef96bf9 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.744823 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822b864-cf31-4e64-a996-8dac3596c19d None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-e91ad89a-13c5-4bc2-8ee5-acd38ef96bf9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:24.766913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:24.768753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:24.770494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:24.770494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:24.770494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:24.770494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:24.770494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:24.770494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:24.770494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:24.770494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:24.770494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:24.770494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:24.775136 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 355/696] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:24.776417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ad89a-13c5-4bc2-8ee5-acd38ef96bf9 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.807207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cbdfa-894e-4f5a-be44-c814c8fd62fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.808164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cbdfa-894e-4f5a-be44-c814c8fd62fc None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.808582 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cbdfa-894e-4f5a-be44-c814c8fd62fc None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.808884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cbdfa-894e-4f5a-be44-c814c8fd62fc None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.809768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cbdfa-894e-4f5a-be44-c814c8fd62fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:24.810158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cbdfa-894e-4f5a-be44-c814c8fd62fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.812496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cbdfa-894e-4f5a-be44-c814c8fd62fc None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.834379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 342/697] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:24.834379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cbdfa-894e-4f5a-be44-c814c8fd62fc None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:24.902750 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:24.912995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ad89a-13c5-4bc2-8ee5-acd38ef96bf9 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.950873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef3b98-7db7-4989-bb23-18f90d8855ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:24.951832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef3b98-7db7-4989-bb23-18f90d8855ee None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:24.952364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef3b98-7db7-4989-bb23-18f90d8855ee None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:24.952810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef3b98-7db7-4989-bb23-18f90d8855ee None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:24.953704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef3b98-7db7-4989-bb23-18f90d8855ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:24.954163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef3b98-7db7-4989-bb23-18f90d8855ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:24.955709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef3b98-7db7-4989-bb23-18f90d8855ee None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:24.969143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 356/698] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:24.969656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cbdfa-894e-4f5a-be44-c814c8fd62fc None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:24.970858 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef3b98-7db7-4989-bb23-18f90d8855ee None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.014850 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac66e30-9982-4f9d-bb5c-c0bc0e2a0616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.015542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac66e30-9982-4f9d-bb5c-c0bc0e2a0616 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.015823 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac66e30-9982-4f9d-bb5c-c0bc0e2a0616 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.016096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac66e30-9982-4f9d-bb5c-c0bc0e2a0616 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.016969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac66e30-9982-4f9d-bb5c-c0bc0e2a0616 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:25.017349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac66e30-9982-4f9d-bb5c-c0bc0e2a0616 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.018826 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac66e30-9982-4f9d-bb5c-c0bc0e2a0616 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.040655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 343/699] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:25.041454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac66e30-9982-4f9d-bb5c-c0bc0e2a0616 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.046279 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:25.119082 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef3b98-7db7-4989-bb23-18f90d8855ee None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.177557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac66e30-9982-4f9d-bb5c-c0bc0e2a0616 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.179942 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cf23b-6d04-4c05-9b0c-11002adcb70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.181118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cf23b-6d04-4c05-9b0c-11002adcb70a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.181632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cf23b-6d04-4c05-9b0c-11002adcb70a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.182116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cf23b-6d04-4c05-9b0c-11002adcb70a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.182703 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:25.191045 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cf23b-6d04-4c05-9b0c-11002adcb70a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.192974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cf23b-6d04-4c05-9b0c-11002adcb70a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-0f2cf23b-6d04-4c05-9b0c-11002adcb70a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:25.209946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:25.211512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:25.212445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:25.212445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:25.212445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:25.212445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:25.212445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:25.212445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:25.212445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:25.212445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:25.214956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 357/700] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:25.216192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cf23b-6d04-4c05-9b0c-11002adcb70a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.236780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df882f-51d8-4908-ab57-7ff447ae250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.237997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df882f-51d8-4908-ab57-7ff447ae250f None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.238668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df882f-51d8-4908-ab57-7ff447ae250f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.239127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df882f-51d8-4908-ab57-7ff447ae250f None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.240570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df882f-51d8-4908-ab57-7ff447ae250f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.243667 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df882f-51d8-4908-ab57-7ff447ae250f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.341796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cf23b-6d04-4c05-9b0c-11002adcb70a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.365054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:25.366278 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 344/701] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:43:25 2022] POST /identity/v3/users => generated 254 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:25.367925 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df882f-51d8-4908-ab57-7ff447ae250f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.390306 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34a64c-35d9-463f-92c9-c92eddc32953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.391430 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34a64c-35d9-463f-92c9-c92eddc32953 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.393737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34a64c-35d9-463f-92c9-c92eddc32953 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.393737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34a64c-35d9-463f-92c9-c92eddc32953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.491584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34a64c-35d9-463f-92c9-c92eddc32953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.493601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34a64c-35d9-463f-92c9-c92eddc32953 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.624425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df882f-51d8-4908-ab57-7ff447ae250f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.624863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 358/702] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:25.630538 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34a64c-35d9-463f-92c9-c92eddc32953 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.653965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:25.693393 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec302e1-6eb9-4f21-9876-fd0ad2516c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.693980 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec302e1-6eb9-4f21-9876-fd0ad2516c7e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.694406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec302e1-6eb9-4f21-9876-fd0ad2516c7e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.696281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec302e1-6eb9-4f21-9876-fd0ad2516c7e None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.697273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec302e1-6eb9-4f21-9876-fd0ad2516c7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:25.697761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec302e1-6eb9-4f21-9876-fd0ad2516c7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.699292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec302e1-6eb9-4f21-9876-fd0ad2516c7e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.713663 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 345/703] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:25.719671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec302e1-6eb9-4f21-9876-fd0ad2516c7e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.740542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:25.771121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34a64c-35d9-463f-92c9-c92eddc32953 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.826362 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6d7b8-78af-4b6a-9e71-7594c5e3dc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.828374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6d7b8-78af-4b6a-9e71-7594c5e3dc09 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.829038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6d7b8-78af-4b6a-9e71-7594c5e3dc09 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.830172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6d7b8-78af-4b6a-9e71-7594c5e3dc09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.843868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6d7b8-78af-4b6a-9e71-7594c5e3dc09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.845628 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6d7b8-78af-4b6a-9e71-7594c5e3dc09 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.856253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec302e1-6eb9-4f21-9876-fd0ad2516c7e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-48c6d7b8-78af-4b6a-9e71-7594c5e3dc09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:25.866365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:25.868728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:25.870980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:25.870980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:25.870980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:25.870980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:25.870980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 359/704] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:25.870980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561f6e2-c5ca-4f6f-92a3-48044b061a71 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.871346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561f6e2-c5ca-4f6f-92a3-48044b061a71 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.871577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561f6e2-c5ca-4f6f-92a3-48044b061a71 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.876801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 360/705] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:25.877466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561f6e2-c5ca-4f6f-92a3-48044b061a71 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:25.906994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0083c6-2a9e-44f0-9e6e-6a56dbd72783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:25.907662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0083c6-2a9e-44f0-9e6e-6a56dbd72783 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.907946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0083c6-2a9e-44f0-9e6e-6a56dbd72783 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.908223 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0083c6-2a9e-44f0-9e6e-6a56dbd72783 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.909047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0083c6-2a9e-44f0-9e6e-6a56dbd72783 None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=1ee9152572224cc499229e0fa2b81ac5) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:25.909363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0083c6-2a9e-44f0-9e6e-6a56dbd72783 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:25.910874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0083c6-2a9e-44f0-9e6e-6a56dbd72783 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:25.918018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 346/706] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/role_assignments?user.id=1ee9152572224cc499229e0fa2b81ac5&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:25.919339 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b65a50-c770-497d-963e-8144ed858ad8 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:25.919627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b65a50-c770-497d-963e-8144ed858ad8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:25.919902 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b65a50-c770-497d-963e-8144ed858ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:25.947465 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-a2b65a50-c770-497d-963e-8144ed858ad8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:25.950054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a2b65a50-c770-497d-963e-8144ed858ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:26.002786 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:26.073928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 347/707] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:26.085532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561f6e2-c5ca-4f6f-92a3-48044b061a71 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.110952 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b65a50-c770-497d-963e-8144ed858ad8 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.127204 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:26.138429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e6374-af7a-4135-9cfb-c2f3cb69efaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.139382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e6374-af7a-4135-9cfb-c2f3cb69efaa None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.139811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e6374-af7a-4135-9cfb-c2f3cb69efaa None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.145265 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e6374-af7a-4135-9cfb-c2f3cb69efaa None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.145265 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e6374-af7a-4135-9cfb-c2f3cb69efaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:26.145265 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e6374-af7a-4135-9cfb-c2f3cb69efaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.145265 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e6374-af7a-4135-9cfb-c2f3cb69efaa None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:26.160678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 361/708] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:26.168957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e6374-af7a-4135-9cfb-c2f3cb69efaa None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.259627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b65a50-c770-497d-963e-8144ed858ad8 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.325961 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e6374-af7a-4135-9cfb-c2f3cb69efaa None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.326901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef1590-4fb7-4b8d-9cd6-796966ab5fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.329139 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef1590-4fb7-4b8d-9cd6-796966ab5fbe None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.329556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef1590-4fb7-4b8d-9cd6-796966ab5fbe None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.329942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef1590-4fb7-4b8d-9cd6-796966ab5fbe None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.331129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef1590-4fb7-4b8d-9cd6-796966ab5fbe None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.332691 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef1590-4fb7-4b8d-9cd6-796966ab5fbe None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-c4ef1590-4fb7-4b8d-9cd6-796966ab5fbe None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:26.354213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:26.360708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:26.362372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:26.366606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:26.367272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 348/709] 10.222.0.17 () {56 vars in 1129 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:26.367272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f072be-3cd4-4d5d-9336-6573dc731044 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.367272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f072be-3cd4-4d5d-9336-6573dc731044 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.367272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f072be-3cd4-4d5d-9336-6573dc731044 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.367272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 349/710] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:26.367272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f072be-3cd4-4d5d-9336-6573dc731044 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.380799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ef012-65ab-4140-bf94-e38930f7cf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.382514 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ef012-65ab-4140-bf94-e38930f7cf2b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.382857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ef012-65ab-4140-bf94-e38930f7cf2b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.383033 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ef012-65ab-4140-bf94-e38930f7cf2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.393239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ef012-65ab-4140-bf94-e38930f7cf2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.395317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ef012-65ab-4140-bf94-e38930f7cf2b None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-df0ef012-65ab-4140-bf94-e38930f7cf2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:26.417134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:26.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:26.420484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:26.420484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:26.420484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:26.420484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:26.420484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:26.420484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:26.420484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:26.420484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:26.420484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 362/711] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:26.426570 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8e208-258b-4361-a85f-e5752aed0b43 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.433679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8e208-258b-4361-a85f-e5752aed0b43 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.433679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8e208-258b-4361-a85f-e5752aed0b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.455421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-bcb8e208-258b-4361-a85f-e5752aed0b43 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:26.457016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-bcb8e208-258b-4361-a85f-e5752aed0b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:26.470959 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:26.516039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f072be-3cd4-4d5d-9336-6573dc731044 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.569425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ecdfe-45c4-4788-8610-f5f0f1117945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.569425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ecdfe-45c4-4788-8610-f5f0f1117945 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.569425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ecdfe-45c4-4788-8610-f5f0f1117945 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.569425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ecdfe-45c4-4788-8610-f5f0f1117945 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.570047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ecdfe-45c4-4788-8610-f5f0f1117945 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:26.570047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ecdfe-45c4-4788-8610-f5f0f1117945 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.570047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ecdfe-45c4-4788-8610-f5f0f1117945 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:26.571054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 363/712] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:26.573225 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8e208-258b-4361-a85f-e5752aed0b43 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.581052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 350/713] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:26.589671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ecdfe-45c4-4788-8610-f5f0f1117945 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.739133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8e208-258b-4361-a85f-e5752aed0b43 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.743986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ecdfe-45c4-4788-8610-f5f0f1117945 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:26.758524 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:26.777278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3654fe-a230-4003-a7f9-616226b5fb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.778173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3654fe-a230-4003-a7f9-616226b5fb03 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.778384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3654fe-a230-4003-a7f9-616226b5fb03 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.778736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3654fe-a230-4003-a7f9-616226b5fb03 None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.779810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3654fe-a230-4003-a7f9-616226b5fb03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:26.779921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3654fe-a230-4003-a7f9-616226b5fb03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.781367 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3654fe-a230-4003-a7f9-616226b5fb03 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:26.794116 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8aa28f-1b38-45d6-a11e-2e2f7380ad0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:26.798511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8aa28f-1b38-45d6-a11e-2e2f7380ad0c None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:26.798673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 364/714] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:26.798673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3654fe-a230-4003-a7f9-616226b5fb03 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.799828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8aa28f-1b38-45d6-a11e-2e2f7380ad0c None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:26.802121 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8aa28f-1b38-45d6-a11e-2e2f7380ad0c None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:26.803212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8aa28f-1b38-45d6-a11e-2e2f7380ad0c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:26.803700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8aa28f-1b38-45d6-a11e-2e2f7380ad0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:26.805155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8aa28f-1b38-45d6-a11e-2e2f7380ad0c None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:26.811942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 351/715] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:26.812882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8aa28f-1b38-45d6-a11e-2e2f7380ad0c None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:26.826915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:27.009403 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3654fe-a230-4003-a7f9-616226b5fb03 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:27.044837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8aa28f-1b38-45d6-a11e-2e2f7380ad0c None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:27.081781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c1eac-481e-4de8-9cb0-79f8e9641981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:27.083528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c1eac-481e-4de8-9cb0-79f8e9641981 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:27.084315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c1eac-481e-4de8-9cb0-79f8e9641981 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:27.085075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c1eac-481e-4de8-9cb0-79f8e9641981 None admin] PATH_INFO: `/v3/services/network` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:27.086823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c1eac-481e-4de8-9cb0-79f8e9641981 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:27.087698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2161b73-94cd-47f7-b273-dbba0988b731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:27.087698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2161b73-94cd-47f7-b273-dbba0988b731 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:27.088380 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2161b73-94cd-47f7-b273-dbba0988b731 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:27.088380 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2161b73-94cd-47f7-b273-dbba0988b731 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:27.089060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2161b73-94cd-47f7-b273-dbba0988b731 None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=e628cb75e0c74f27a390945c8a04335c) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:27.089888 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2161b73-94cd-47f7-b273-dbba0988b731 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:27.091374 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2161b73-94cd-47f7-b273-dbba0988b731 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:27.092877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c1eac-481e-4de8-9cb0-79f8e9641981 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:27.098583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 352/716] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/role_assignments?user.id=e628cb75e0c74f27a390945c8a04335c&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-827c1eac-481e-4de8-9cb0-79f8e9641981 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:27.126003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:27.127957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:27.129562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:27.135507 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 365/717] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:26 2022] GET /identity/v3/services/network => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:27.143515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2161b73-94cd-47f7-b273-dbba0988b731 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:27.250380 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:27.298775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2161b73-94cd-47f7-b273-dbba0988b731 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:27.338624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b4a56-a1f1-4f23-9737-e1ea353562dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:27.339318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b4a56-a1f1-4f23-9737-e1ea353562dd None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:27.339610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b4a56-a1f1-4f23-9737-e1ea353562dd None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:27.339894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b4a56-a1f1-4f23-9737-e1ea353562dd None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:27.340688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b4a56-a1f1-4f23-9737-e1ea353562dd None admin] RBAC: Adding query filter params (name=network) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:27.341004 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b4a56-a1f1-4f23-9737-e1ea353562dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:27.342422 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b4a56-a1f1-4f23-9737-e1ea353562dd None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:27.347902 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 353/718] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:27.353490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c1eac-481e-4de8-9cb0-79f8e9641981 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:27.440866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:27.514890 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c1eac-481e-4de8-9cb0-79f8e9641981 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:27.558573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713dda2-ec72-45e9-810a-c29419d221e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:27.559288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713dda2-ec72-45e9-810a-c29419d221e6 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:27.559577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713dda2-ec72-45e9-810a-c29419d221e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:27.559854 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713dda2-ec72-45e9-810a-c29419d221e6 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:27.560569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713dda2-ec72-45e9-810a-c29419d221e6 None admin] RBAC: Adding query filter params (type=network) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:27.560877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713dda2-ec72-45e9-810a-c29419d221e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:27.562258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713dda2-ec72-45e9-810a-c29419d221e6 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:27.569306 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 366/719] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:28.346122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31a82b-03ec-4878-881f-5ff1e716bcc1 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:28.346762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31a82b-03ec-4878-881f-5ff1e716bcc1 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:28.346854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31a82b-03ec-4878-881f-5ff1e716bcc1 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:28.349046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 354/720] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:28.357327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d87b2e-c016-4a43-ba3f-cac83d13a7b3 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:28.357908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d87b2e-c016-4a43-ba3f-cac83d13a7b3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:28.358319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d87b2e-c016-4a43-ba3f-cac83d13a7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:28.397921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-91d87b2e-c016-4a43-ba3f-cac83d13a7b3 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:28.400581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-91d87b2e-c016-4a43-ba3f-cac83d13a7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:28.519428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 367/721] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:28.556683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31a82b-03ec-4878-881f-5ff1e716bcc1 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:28.707082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31a82b-03ec-4878-881f-5ff1e716bcc1 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:28.752994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:28.759816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dc6ae-f56d-452d-9ef4-a84e9df25e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:28.760827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dc6ae-f56d-452d-9ef4-a84e9df25e24 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:28.761115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dc6ae-f56d-452d-9ef4-a84e9df25e24 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:28.761393 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dc6ae-f56d-452d-9ef4-a84e9df25e24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:28.782721 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3dc6ae-f56d-452d-9ef4-a84e9df25e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-7f3dc6ae-f56d-452d-9ef4-a84e9df25e24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:28.795163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:28.796230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:28.797127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:28.797127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:28.797127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:28.797127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:28.797127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:28.797127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:28.797127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:28.797127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:28.797127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:28.797127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:28.797127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:28.797127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:28.797127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 355/722] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:28.802426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d87b2e-c016-4a43-ba3f-cac83d13a7b3 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:28.854358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:28.946193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d87b2e-c016-4a43-ba3f-cac83d13a7b3 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:29.005002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07e923-e671-49ef-b474-b86b63d98bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:29.006093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07e923-e671-49ef-b474-b86b63d98bbd None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.006613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07e923-e671-49ef-b474-b86b63d98bbd None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.007105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07e923-e671-49ef-b474-b86b63d98bbd None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.008993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07e923-e671-49ef-b474-b86b63d98bbd None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:29.009654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07e923-e671-49ef-b474-b86b63d98bbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:29.011510 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07e923-e671-49ef-b474-b86b63d98bbd None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:29.024697 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e65ebe-e6dc-4993-9d24-e43c22d76aeb None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.025040 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e65ebe-e6dc-4993-9d24-e43c22d76aeb None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.025272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e65ebe-e6dc-4993-9d24-e43c22d76aeb None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.027124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 356/723] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:29.029020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 368/724] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:29.032634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5b6d7-e46e-4666-a32d-8b93111ed009 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.033301 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5b6d7-e46e-4666-a32d-8b93111ed009 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.034669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07e923-e671-49ef-b474-b86b63d98bbd None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:29.035658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5b6d7-e46e-4666-a32d-8b93111ed009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.083580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-f6d5b6d7-e46e-4666-a32d-8b93111ed009 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:29.087951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f6d5b6d7-e46e-4666-a32d-8b93111ed009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:29.229384 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 357/725] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 18:43:29.243210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:29.248222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07e923-e671-49ef-b474-b86b63d98bbd None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:29.274637 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5b6d7-e46e-4666-a32d-8b93111ed009 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:29.430815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5b6d7-e46e-4666-a32d-8b93111ed009 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:29.447787 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511038a-001c-42a1-8841-6074b9407891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:29.448612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511038a-001c-42a1-8841-6074b9407891 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.448896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511038a-001c-42a1-8841-6074b9407891 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.449172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511038a-001c-42a1-8841-6074b9407891 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.467569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511038a-001c-42a1-8841-6074b9407891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:29.469047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511038a-001c-42a1-8841-6074b9407891 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:29.484625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:29.495841 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ada81-72d7-4052-95f5-3a9a986d775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:29.500113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ada81-72d7-4052-95f5-3a9a986d775b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.500113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ada81-72d7-4052-95f5-3a9a986d775b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.500113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ada81-72d7-4052-95f5-3a9a986d775b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-d511038a-001c-42a1-8841-6074b9407891 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:29.503288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:29.504165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:29.505050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:29.505050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:29.505050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:29.505050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:29.510129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ada81-72d7-4052-95f5-3a9a986d775b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:29.514891 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 369/726] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 481 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-668ada81-72d7-4052-95f5-3a9a986d775b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:29.522147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:29.524034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:29.524923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:29.524923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:29.524923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:29.524923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:29.524923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:29.524923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:29.524923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:29.524923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:29.524923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:29.524923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:29.524923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:29.524923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:29.526927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511038a-001c-42a1-8841-6074b9407891 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:29.527077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 358/727] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:29.532370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ada81-72d7-4052-95f5-3a9a986d775b None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:29.659780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ada81-72d7-4052-95f5-3a9a986d775b None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:29.667226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511038a-001c-42a1-8841-6074b9407891 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:29.705311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4215e-fa5e-4dbc-b950-b72d4a9ef822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:29.705877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100b04e-2c4e-42ec-9943-763156c2de81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:29.706588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100b04e-2c4e-42ec-9943-763156c2de81 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.706879 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100b04e-2c4e-42ec-9943-763156c2de81 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.707117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4215e-fa5e-4dbc-b950-b72d4a9ef822 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.707269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100b04e-2c4e-42ec-9943-763156c2de81 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.707453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4215e-fa5e-4dbc-b950-b72d4a9ef822 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.707685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4215e-fa5e-4dbc-b950-b72d4a9ef822 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.707903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100b04e-2c4e-42ec-9943-763156c2de81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:29.708216 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100b04e-2c4e-42ec-9943-763156c2de81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:29.708438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4215e-fa5e-4dbc-b950-b72d4a9ef822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:29.708744 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4215e-fa5e-4dbc-b950-b72d4a9ef822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:29.709766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100b04e-2c4e-42ec-9943-763156c2de81 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:29.710091 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4215e-fa5e-4dbc-b950-b72d4a9ef822 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:29.723604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 370/728] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:29.729339 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52d241-8831-4b16-8ce3-9f5af6d391e7 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.729339 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52d241-8831-4b16-8ce3-9f5af6d391e7 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.729339 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52d241-8831-4b16-8ce3-9f5af6d391e7 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.729627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 359/729] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:29.729627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e1469-6357-4b24-85a7-ee8a44ae761a None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.729627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e1469-6357-4b24-85a7-ee8a44ae761a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.729627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e1469-6357-4b24-85a7-ee8a44ae761a None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.730581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 360/730] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:29.731198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e1469-6357-4b24-85a7-ee8a44ae761a None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:29.731629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 371/731] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:29.732456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52d241-8831-4b16-8ce3-9f5af6d391e7 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:29.746193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:29.781022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:29.861111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52d241-8831-4b16-8ce3-9f5af6d391e7 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:29.865218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e1469-6357-4b24-85a7-ee8a44ae761a None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:29.903509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd85b9a8-c873-46fe-a286-9a66e939d37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:29.904354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85b9a8-c873-46fe-a286-9a66e939d37c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.904670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85b9a8-c873-46fe-a286-9a66e939d37c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.904944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85b9a8-c873-46fe-a286-9a66e939d37c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.905493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa55001-9428-48bf-bf62-17e1da893feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:29.906410 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa55001-9428-48bf-bf62-17e1da893feb None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.906725 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa55001-9428-48bf-bf62-17e1da893feb None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.906981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa55001-9428-48bf-bf62-17e1da893feb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.914086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd85b9a8-c873-46fe-a286-9a66e939d37c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:29.915890 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd85b9a8-c873-46fe-a286-9a66e939d37c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:29.921133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa55001-9428-48bf-bf62-17e1da893feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:29.922697 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa55001-9428-48bf-bf62-17e1da893feb None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-dd85b9a8-c873-46fe-a286-9a66e939d37c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:29.933508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:29.941387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:29.942527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:29.942527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:29.942527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:29.942527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:29.942527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:29.942527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:29.942527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:29.942527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:29.942527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:29.942527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:29.942527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 372/732] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 18:43:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:29.942527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86e758-9e2b-423a-bdc3-ec170b65b634 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.942527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86e758-9e2b-423a-bdc3-ec170b65b634 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.943583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86e758-9e2b-423a-bdc3-ec170b65b634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-bfa55001-9428-48bf-bf62-17e1da893feb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:29.957489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:29.958546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:29.965466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:29.965466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:29.965466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:29.965466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:29.965466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 361/733] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:29.965803 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ce86e758-9e2b-423a-bdc3-ec170b65b634 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:29.965897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06895e-7728-4333-8f64-8f94888fee29 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:29.965897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06895e-7728-4333-8f64-8f94888fee29 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:29.965897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06895e-7728-4333-8f64-8f94888fee29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:29.966099 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ce86e758-9e2b-423a-bdc3-ec170b65b634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:29.982273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-7e06895e-7728-4333-8f64-8f94888fee29 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:29.983499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-7e06895e-7728-4333-8f64-8f94888fee29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:30.056953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.062660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 373/734] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:30.063761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86e758-9e2b-423a-bdc3-ec170b65b634 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.082494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 362/735] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:30.083810 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06895e-7728-4333-8f64-8f94888fee29 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.128333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.217921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86e758-9e2b-423a-bdc3-ec170b65b634 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.237734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06895e-7728-4333-8f64-8f94888fee29 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.279981 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92aaa2-33da-4e8a-934f-d0fe7f0ac4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.281085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92aaa2-33da-4e8a-934f-d0fe7f0ac4d2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.281621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92aaa2-33da-4e8a-934f-d0fe7f0ac4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.282104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92aaa2-33da-4e8a-934f-d0fe7f0ac4d2 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.283263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92aaa2-33da-4e8a-934f-d0fe7f0ac4d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:30.283657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafad85-0617-44ba-849f-1e5d157b1d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.284019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92aaa2-33da-4e8a-934f-d0fe7f0ac4d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.284656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafad85-0617-44ba-849f-1e5d157b1d94 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.285015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafad85-0617-44ba-849f-1e5d157b1d94 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.285355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafad85-0617-44ba-849f-1e5d157b1d94 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.286093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafad85-0617-44ba-849f-1e5d157b1d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:30.286435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafad85-0617-44ba-849f-1e5d157b1d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.286675 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92aaa2-33da-4e8a-934f-d0fe7f0ac4d2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.287813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafad85-0617-44ba-849f-1e5d157b1d94 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.300567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 363/736] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:30.301464 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafad85-0617-44ba-849f-1e5d157b1d94 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.311270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 374/737] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:30.312221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92aaa2-33da-4e8a-934f-d0fe7f0ac4d2 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.389060 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.403635 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.421868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafad85-0617-44ba-849f-1e5d157b1d94 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.433652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92aaa2-33da-4e8a-934f-d0fe7f0ac4d2 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.465727 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23425c51-cfec-4528-83d9-a5eac3875dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.466568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23425c51-cfec-4528-83d9-a5eac3875dd8 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.466859 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23425c51-cfec-4528-83d9-a5eac3875dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.467143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23425c51-cfec-4528-83d9-a5eac3875dd8 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.468144 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23425c51-cfec-4528-83d9-a5eac3875dd8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.469599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23425c51-cfec-4528-83d9-a5eac3875dd8 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.471259 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2a69a-fb7b-4ea5-b378-1b9bef0ad312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.471866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2a69a-fb7b-4ea5-b378-1b9bef0ad312 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.472156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2a69a-fb7b-4ea5-b378-1b9bef0ad312 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.472156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2a69a-fb7b-4ea5-b378-1b9bef0ad312 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.484859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2a69a-fb7b-4ea5-b378-1b9bef0ad312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-92b2a69a-fb7b-4ea5-b378-1b9bef0ad312 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:30.493083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:30.494115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:30.494995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:30.494995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:30.494995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:30.494995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:30.494995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:30.494995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:30.494995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:30.494995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:30.494995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:30.494995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:30.494995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:30.494995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:30.494995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 375/738] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:30.497205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2a69a-fb7b-4ea5-b378-1b9bef0ad312 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.552883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 364/739] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:43:30 2022] POST /identity/v3/services => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:30.555396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23425c51-cfec-4528-83d9-a5eac3875dd8 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.609110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2a69a-fb7b-4ea5-b378-1b9bef0ad312 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.631724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.660445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdec7cb-8bc5-4fcf-ac6b-4c0a12fde580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.661438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdec7cb-8bc5-4fcf-ac6b-4c0a12fde580 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.661827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdec7cb-8bc5-4fcf-ac6b-4c0a12fde580 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.662752 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdec7cb-8bc5-4fcf-ac6b-4c0a12fde580 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.672538 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdec7cb-8bc5-4fcf-ac6b-4c0a12fde580 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.674313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdec7cb-8bc5-4fcf-ac6b-4c0a12fde580 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-5cdec7cb-8bc5-4fcf-ac6b-4c0a12fde580 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:30.691377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:30.692395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:30.693506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:30.693506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:30.693506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:30.693506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:30.693506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:30.693506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:30.693506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:30.693506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:30.693506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:30.693506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:30.694202 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23425c51-cfec-4528-83d9-a5eac3875dd8 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.694293 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 376/740] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:30.694293 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdec7cb-8bc5-4fcf-ac6b-4c0a12fde580 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.742613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfff9e-7a54-4aef-9fbb-97e5cbffef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.743957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfff9e-7a54-4aef-9fbb-97e5cbffef61 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.743957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfff9e-7a54-4aef-9fbb-97e5cbffef61 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.743957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfff9e-7a54-4aef-9fbb-97e5cbffef61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.763354 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.823276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdec7cb-8bc5-4fcf-ac6b-4c0a12fde580 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:30.823690 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfff9e-7a54-4aef-9fbb-97e5cbffef61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.826448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfff9e-7a54-4aef-9fbb-97e5cbffef61 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.859664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07ce0a-5b0e-4550-b4cc-ba9b6b06e9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:30.860344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07ce0a-5b0e-4550-b4cc-ba9b6b06e9b0 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.860627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07ce0a-5b0e-4550-b4cc-ba9b6b06e9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.860897 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07ce0a-5b0e-4550-b4cc-ba9b6b06e9b0 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.861651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07ce0a-5b0e-4550-b4cc-ba9b6b06e9b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:30.861959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07ce0a-5b0e-4550-b4cc-ba9b6b06e9b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:30.863362 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07ce0a-5b0e-4550-b4cc-ba9b6b06e9b0 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:30.877197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 377/741] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:30.877197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414090e2-71a5-46f1-96b2-e00aea8ff81c None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:30.877197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414090e2-71a5-46f1-96b2-e00aea8ff81c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:30.877197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414090e2-71a5-46f1-96b2-e00aea8ff81c None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:30.877197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 378/742] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:30.877197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414090e2-71a5-46f1-96b2-e00aea8ff81c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:30.922317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:30.950254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 365/743] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:30.951406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfff9e-7a54-4aef-9fbb-97e5cbffef61 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.001874 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414090e2-71a5-46f1-96b2-e00aea8ff81c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.004233 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:31.048172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8b7e2-80e7-4515-8560-4013154348f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.048964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8b7e2-80e7-4515-8560-4013154348f8 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.049254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8b7e2-80e7-4515-8560-4013154348f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.049533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8b7e2-80e7-4515-8560-4013154348f8 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.050726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8b7e2-80e7-4515-8560-4013154348f8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:31.051043 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8b7e2-80e7-4515-8560-4013154348f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.052623 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8b7e2-80e7-4515-8560-4013154348f8 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.071970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfff9e-7a54-4aef-9fbb-97e5cbffef61 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.075074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 379/744] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:31.076431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70501e-2fdf-4ec3-b812-1c17a8ab47ae None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.076783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70501e-2fdf-4ec3-b812-1c17a8ab47ae None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.077062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70501e-2fdf-4ec3-b812-1c17a8ab47ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.096233 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-cd70501e-2fdf-4ec3-b812-1c17a8ab47ae None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:31.097437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-cd70501e-2fdf-4ec3-b812-1c17a8ab47ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:31.116571 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e3ec8-1279-4f97-98c6-66015da7e6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.118070 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e3ec8-1279-4f97-98c6-66015da7e6da None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.118404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e3ec8-1279-4f97-98c6-66015da7e6da None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.118799 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e3ec8-1279-4f97-98c6-66015da7e6da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.137969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e3ec8-1279-4f97-98c6-66015da7e6da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.139899 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e3ec8-1279-4f97-98c6-66015da7e6da None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-538e3ec8-1279-4f97-98c6-66015da7e6da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:31.160443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:31.161499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:31.170515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:31.170515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:31.170515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:31.170515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:31.170515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 366/745] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:31.170515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e3ec8-1279-4f97-98c6-66015da7e6da None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.192369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 380/746] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 18:43:31.193665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70501e-2fdf-4ec3-b812-1c17a8ab47ae None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.222451 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:31.310268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:31.313278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70501e-2fdf-4ec3-b812-1c17a8ab47ae None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.318745 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e3ec8-1279-4f97-98c6-66015da7e6da None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.358517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779f7b7-bfb4-4dff-b43f-d2f3d3edd5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.359194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779f7b7-bfb4-4dff-b43f-d2f3d3edd5e4 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.359481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779f7b7-bfb4-4dff-b43f-d2f3d3edd5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.359759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779f7b7-bfb4-4dff-b43f-d2f3d3edd5e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.363878 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb583e0-3581-4be6-951a-422c798b1175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.364875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb583e0-3581-4be6-951a-422c798b1175 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.365437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb583e0-3581-4be6-951a-422c798b1175 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.366568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb583e0-3581-4be6-951a-422c798b1175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.381062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb583e0-3581-4be6-951a-422c798b1175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.383417 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb583e0-3581-4be6-951a-422c798b1175 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-2cb583e0-3581-4be6-951a-422c798b1175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:31.404127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:31.405997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:31.407124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:31.407124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:31.407124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:31.407124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:31.407124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 367/747] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:31.407124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb583e0-3581-4be6-951a-422c798b1175 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.455002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779f7b7-bfb4-4dff-b43f-d2f3d3edd5e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.456494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779f7b7-bfb4-4dff-b43f-d2f3d3edd5e4 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.513998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:31.564210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb583e0-3581-4be6-951a-422c798b1175 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.604928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b28de-e3a1-4196-bdfa-c935cf3f9d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.605898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b28de-e3a1-4196-bdfa-c935cf3f9d68 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.606254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b28de-e3a1-4196-bdfa-c935cf3f9d68 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.606623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b28de-e3a1-4196-bdfa-c935cf3f9d68 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.607849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b28de-e3a1-4196-bdfa-c935cf3f9d68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:31.607849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b28de-e3a1-4196-bdfa-c935cf3f9d68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.609465 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b28de-e3a1-4196-bdfa-c935cf3f9d68 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.631477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 368/748] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:31.631477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b28de-e3a1-4196-bdfa-c935cf3f9d68 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.631762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 381/749] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 18:43:31.631762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779f7b7-bfb4-4dff-b43f-d2f3d3edd5e4 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.644578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:31.750928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779f7b7-bfb4-4dff-b43f-d2f3d3edd5e4 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.754031 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b28de-e3a1-4196-bdfa-c935cf3f9d68 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.780142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:31.792782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870db33-4721-4863-9952-b892ec82a050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.797197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870db33-4721-4863-9952-b892ec82a050 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.797197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870db33-4721-4863-9952-b892ec82a050 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.797197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870db33-4721-4863-9952-b892ec82a050 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.797197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870db33-4721-4863-9952-b892ec82a050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:31.797197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870db33-4721-4863-9952-b892ec82a050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.797197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870db33-4721-4863-9952-b892ec82a050 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.797629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee31cf-56dc-45f8-8868-1840a23631e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.797888 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee31cf-56dc-45f8-8868-1840a23631e9 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.798518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee31cf-56dc-45f8-8868-1840a23631e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.799460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee31cf-56dc-45f8-8868-1840a23631e9 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.800182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee31cf-56dc-45f8-8868-1840a23631e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.802471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee31cf-56dc-45f8-8868-1840a23631e9 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.810925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 382/750] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:31.811788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870db33-4721-4863-9952-b892ec82a050 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:31.920606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870db33-4721-4863-9952-b892ec82a050 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:31.955969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb031f9-f60c-427f-afb7-3637f087c1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:31.956790 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb031f9-f60c-427f-afb7-3637f087c1d0 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:31.957076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb031f9-f60c-427f-afb7-3637f087c1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:31.957356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb031f9-f60c-427f-afb7-3637f087c1d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:31.965152 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb031f9-f60c-427f-afb7-3637f087c1d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:31.966627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb031f9-f60c-427f-afb7-3637f087c1d0 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-8fb031f9-f60c-427f-afb7-3637f087c1d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:31.981069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:31.982086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:31.983081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:31.983081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:31.983081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:31.983081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:31.983081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:31.983081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:31.983081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:31.983081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:31.983081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:31.983081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:31.983081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 383/751] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:31.983679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb031f9-f60c-427f-afb7-3637f087c1d0 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.031650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.087294 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb031f9-f60c-427f-afb7-3637f087c1d0 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.122247 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120fcc8-0bd0-4077-9219-0ee842a198f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.123086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120fcc8-0bd0-4077-9219-0ee842a198f2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.123374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120fcc8-0bd0-4077-9219-0ee842a198f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.123795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120fcc8-0bd0-4077-9219-0ee842a198f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.136407 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120fcc8-0bd0-4077-9219-0ee842a198f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.137827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120fcc8-0bd0-4077-9219-0ee842a198f2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-d120fcc8-0bd0-4077-9219-0ee842a198f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:32.155735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:32.156750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:32.169621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:32.169621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:32.169621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:32.169621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:32.169621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 384/752] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:32.169621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120fcc8-0bd0-4077-9219-0ee842a198f2 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.207746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 369/753] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:43:31 2022] POST /identity/v3/services => generated 240 bytes in 580 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:32.208343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee31cf-56dc-45f8-8868-1840a23631e9 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.279839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.289473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120fcc8-0bd0-4077-9219-0ee842a198f2 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.317081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.343965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78329b71-72e3-4165-9b3c-3e543519085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.345226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78329b71-72e3-4165-9b3c-3e543519085f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.345716 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78329b71-72e3-4165-9b3c-3e543519085f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.346181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78329b71-72e3-4165-9b3c-3e543519085f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.356555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78329b71-72e3-4165-9b3c-3e543519085f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.358603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78329b71-72e3-4165-9b3c-3e543519085f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.364002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee31cf-56dc-45f8-8868-1840a23631e9 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-78329b71-72e3-4165-9b3c-3e543519085f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:32.375985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:32.376953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:32.377883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:32.377883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:32.377883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:32.377883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:32.377883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:32.377883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:32.377883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:32.377883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:32.379942 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 385/754] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:32.380973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78329b71-72e3-4165-9b3c-3e543519085f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.416770 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a41ec-5ccc-4d87-8cb0-647bfdbe28a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.417615 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a41ec-5ccc-4d87-8cb0-647bfdbe28a5 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.417903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a41ec-5ccc-4d87-8cb0-647bfdbe28a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.418184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a41ec-5ccc-4d87-8cb0-647bfdbe28a5 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.420125 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a41ec-5ccc-4d87-8cb0-647bfdbe28a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:32.421076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a41ec-5ccc-4d87-8cb0-647bfdbe28a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.422656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a41ec-5ccc-4d87-8cb0-647bfdbe28a5 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.447643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 370/755] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:32.448820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a41ec-5ccc-4d87-8cb0-647bfdbe28a5 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.515168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78329b71-72e3-4165-9b3c-3e543519085f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.554147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4942ded-f870-4022-b7ef-312a7887d414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.555761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4942ded-f870-4022-b7ef-312a7887d414 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.556348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4942ded-f870-4022-b7ef-312a7887d414 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.556837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4942ded-f870-4022-b7ef-312a7887d414 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.557769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4942ded-f870-4022-b7ef-312a7887d414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:32.558291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4942ded-f870-4022-b7ef-312a7887d414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.559991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4942ded-f870-4022-b7ef-312a7887d414 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.574229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 386/756] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:32.575448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4942ded-f870-4022-b7ef-312a7887d414 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.587198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a41ec-5ccc-4d87-8cb0-647bfdbe28a5 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.602108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.620511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.630370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1ddbd-488b-4aa0-81cc-8dc53ae04a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.631625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1ddbd-488b-4aa0-81cc-8dc53ae04a0e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.631625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1ddbd-488b-4aa0-81cc-8dc53ae04a0e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.631625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1ddbd-488b-4aa0-81cc-8dc53ae04a0e None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.633172 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1ddbd-488b-4aa0-81cc-8dc53ae04a0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:32.633644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1ddbd-488b-4aa0-81cc-8dc53ae04a0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.635409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1ddbd-488b-4aa0-81cc-8dc53ae04a0e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.649021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 371/757] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:32.650035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1ddbd-488b-4aa0-81cc-8dc53ae04a0e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.755377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4942ded-f870-4022-b7ef-312a7887d414 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.794673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1ddbd-488b-4aa0-81cc-8dc53ae04a0e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:32.800322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476d91e-d8e1-403c-9ff4-63ffd5650322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.801192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476d91e-d8e1-403c-9ff4-63ffd5650322 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.801485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476d91e-d8e1-403c-9ff4-63ffd5650322 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.801768 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476d91e-d8e1-403c-9ff4-63ffd5650322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.815974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476d91e-d8e1-403c-9ff4-63ffd5650322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.817643 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476d91e-d8e1-403c-9ff4-63ffd5650322 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.838020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed61dfe-52c0-40bf-839b-f44c93486296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:32.839210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed61dfe-52c0-40bf-839b-f44c93486296 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-4476d91e-d8e1-403c-9ff4-63ffd5650322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:32.839641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:32.840609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:32.841569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:32.841569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:32.841569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:32.841569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:32.841569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 387/758] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:32.841850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476d91e-d8e1-403c-9ff4-63ffd5650322 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.842051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed61dfe-52c0-40bf-839b-f44c93486296 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.842599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed61dfe-52c0-40bf-839b-f44c93486296 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.854648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed61dfe-52c0-40bf-839b-f44c93486296 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:32.856261 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed61dfe-52c0-40bf-839b-f44c93486296 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-2ed61dfe-52c0-40bf-839b-f44c93486296 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:32.872793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:32.873747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:32.874650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:32.874650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:32.874650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:32.874650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:32.874650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:32.874650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:32.874650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:32.874650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:32.878967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 372/759] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:32.880411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370bfeb-360d-4936-a511-d541ef70c323 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:32.880705 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370bfeb-360d-4936-a511-d541ef70c323 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:32.881059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370bfeb-360d-4936-a511-d541ef70c323 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:32.882458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 373/760] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:32.883167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370bfeb-360d-4936-a511-d541ef70c323 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:32.926477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.946093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:32.973376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476d91e-d8e1-403c-9ff4-63ffd5650322 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.016698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370bfeb-360d-4936-a511-d541ef70c323 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.019963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61dc3f-7225-417a-8340-bd8a4ee7f8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.020794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61dc3f-7225-417a-8340-bd8a4ee7f8ef None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.021184 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61dc3f-7225-417a-8340-bd8a4ee7f8ef None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.021562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61dc3f-7225-417a-8340-bd8a4ee7f8ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.022588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61dc3f-7225-417a-8340-bd8a4ee7f8ef None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=c6a88af6e45549f1ab00036923f6ba04) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:33.023805 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61dc3f-7225-417a-8340-bd8a4ee7f8ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.025114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61dc3f-7225-417a-8340-bd8a4ee7f8ef None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.038384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 388/761] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/role_assignments?user.id=c6a88af6e45549f1ab00036923f6ba04&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:33.038384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61dc3f-7225-417a-8340-bd8a4ee7f8ef None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.087570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddd881-1690-42b0-8084-a197ee1938bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.088969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddd881-1690-42b0-8084-a197ee1938bd None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.089566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddd881-1690-42b0-8084-a197ee1938bd None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.090182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddd881-1690-42b0-8084-a197ee1938bd None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.091875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddd881-1690-42b0-8084-a197ee1938bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:33.092516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddd881-1690-42b0-8084-a197ee1938bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.095611 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddd881-1690-42b0-8084-a197ee1938bd None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.113760 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 374/762] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:33.115347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82cce5-3942-4557-a72d-c72e9d9fae6c None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.115659 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82cce5-3942-4557-a72d-c72e9d9fae6c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.115942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82cce5-3942-4557-a72d-c72e9d9fae6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.140864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3e82cce5-3942-4557-a72d-c72e9d9fae6c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:33.142216 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3e82cce5-3942-4557-a72d-c72e9d9fae6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:33.200978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61dc3f-7225-417a-8340-bd8a4ee7f8ef None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.239902 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 375/763] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:33.239902 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82cce5-3942-4557-a72d-c72e9d9fae6c None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.255923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:33.266430 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f5ba1-af8c-438d-addf-85236b95ca17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.267424 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f5ba1-af8c-438d-addf-85236b95ca17 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.267898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f5ba1-af8c-438d-addf-85236b95ca17 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.268347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f5ba1-af8c-438d-addf-85236b95ca17 None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.269286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f5ba1-af8c-438d-addf-85236b95ca17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:33.269745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f5ba1-af8c-438d-addf-85236b95ca17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.271421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f5ba1-af8c-438d-addf-85236b95ca17 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.285666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 389/764] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:33.288316 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f5ba1-af8c-438d-addf-85236b95ca17 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.300907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:33.499187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f5ba1-af8c-438d-addf-85236b95ca17 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.511609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82cce5-3942-4557-a72d-c72e9d9fae6c None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.548473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98884c-e045-400f-ab66-ab590434385c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.549458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98884c-e045-400f-ab66-ab590434385c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.549612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98884c-e045-400f-ab66-ab590434385c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.549886 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98884c-e045-400f-ab66-ab590434385c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.550609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98884c-e045-400f-ab66-ab590434385c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.552078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98884c-e045-400f-ab66-ab590434385c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-5b98884c-e045-400f-ab66-ab590434385c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:33.566722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:33.568498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:33.570054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 390/765] 10.222.0.17 () {56 vars in 1129 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:33.571426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98884c-e045-400f-ab66-ab590434385c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.578392 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daae6cb-5363-4512-9a42-e1b7031d63a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.578392 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daae6cb-5363-4512-9a42-e1b7031d63a8 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.578392 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daae6cb-5363-4512-9a42-e1b7031d63a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.578392 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daae6cb-5363-4512-9a42-e1b7031d63a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.591227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daae6cb-5363-4512-9a42-e1b7031d63a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.593041 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daae6cb-5363-4512-9a42-e1b7031d63a8 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-5daae6cb-5363-4512-9a42-e1b7031d63a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:33.611629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:33.612855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:33.614167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:33.614167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:33.614167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:33.614167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:33.614167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:33.614167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:33.614167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:33.614167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:33.614167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 376/766] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:33.614879 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daae6cb-5363-4512-9a42-e1b7031d63a8 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.660434 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:33.718068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98884c-e045-400f-ab66-ab590434385c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.750176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:33.758291 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daae6cb-5363-4512-9a42-e1b7031d63a8 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:33.760587 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae81dab-8cd5-4755-8abf-adbd589cf622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.761470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae81dab-8cd5-4755-8abf-adbd589cf622 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.761969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae81dab-8cd5-4755-8abf-adbd589cf622 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.769863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae81dab-8cd5-4755-8abf-adbd589cf622 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.769863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae81dab-8cd5-4755-8abf-adbd589cf622 None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=1ee9152572224cc499229e0fa2b81ac5) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:33.769863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae81dab-8cd5-4755-8abf-adbd589cf622 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.769863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae81dab-8cd5-4755-8abf-adbd589cf622 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.777435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 391/767] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/role_assignments?user.id=1ee9152572224cc499229e0fa2b81ac5&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:33.778728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae81dab-8cd5-4755-8abf-adbd589cf622 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.822972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b3fac-c40b-4399-8e8a-46010bbbf0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:33.823935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b3fac-c40b-4399-8e8a-46010bbbf0dd None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:33.824457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b3fac-c40b-4399-8e8a-46010bbbf0dd None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:33.824969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b3fac-c40b-4399-8e8a-46010bbbf0dd None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:33.826418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b3fac-c40b-4399-8e8a-46010bbbf0dd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:33.827325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b3fac-c40b-4399-8e8a-46010bbbf0dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:33.838357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b3fac-c40b-4399-8e8a-46010bbbf0dd None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:33.838357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 377/768] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:33.842376 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b3fac-c40b-4399-8e8a-46010bbbf0dd None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:33.987930 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b3fac-c40b-4399-8e8a-46010bbbf0dd None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:34.028560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae81dab-8cd5-4755-8abf-adbd589cf622 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:34.037690 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9576b41-b35e-4630-b7d1-5ec1d25e38f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:34.038550 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9576b41-b35e-4630-b7d1-5ec1d25e38f2 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.039691 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9576b41-b35e-4630-b7d1-5ec1d25e38f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.040185 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9576b41-b35e-4630-b7d1-5ec1d25e38f2 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.041034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9576b41-b35e-4630-b7d1-5ec1d25e38f2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:34.041354 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9576b41-b35e-4630-b7d1-5ec1d25e38f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:34.042972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9576b41-b35e-4630-b7d1-5ec1d25e38f2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:34.049952 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 378/769] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:34.057348 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9576b41-b35e-4630-b7d1-5ec1d25e38f2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:34.065281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:34.110392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5654769-7557-462f-a876-f9a694f93909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:34.111163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5654769-7557-462f-a876-f9a694f93909 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.111451 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5654769-7557-462f-a876-f9a694f93909 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.111746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5654769-7557-462f-a876-f9a694f93909 None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.112591 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5654769-7557-462f-a876-f9a694f93909 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:34.112905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5654769-7557-462f-a876-f9a694f93909 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:34.114370 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5654769-7557-462f-a876-f9a694f93909 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:34.134802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:34.144402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 392/770] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:34.152499 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5654769-7557-462f-a876-f9a694f93909 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:34.283299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5654769-7557-462f-a876-f9a694f93909 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:34.311671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9576b41-b35e-4630-b7d1-5ec1d25e38f2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:34.338360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1f25a-a941-461f-bd2f-f05deb9028cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:34.339731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1f25a-a941-461f-bd2f-f05deb9028cd None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.340234 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1f25a-a941-461f-bd2f-f05deb9028cd None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.340714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1f25a-a941-461f-bd2f-f05deb9028cd None admin] PATH_INFO: `/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/e628cb75e0c74f27a390945c8a04335c/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.343414 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1f25a-a941-461f-bd2f-f05deb9028cd None admin] RBAC: Authorizing `identity:create_grant(project_id=5d7c3b139f284435aea4ff08a2f0952b, user_id=e628cb75e0c74f27a390945c8a04335c, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:34.345540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1f25a-a941-461f-bd2f-f05deb9028cd None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:34.386681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5e9be-4df7-47b0-9dad-d4fe738153f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:34.387428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5e9be-4df7-47b0-9dad-d4fe738153f0 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.387724 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5e9be-4df7-47b0-9dad-d4fe738153f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.388003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5e9be-4df7-47b0-9dad-d4fe738153f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.388981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5e9be-4df7-47b0-9dad-d4fe738153f0 None admin] RBAC: Adding query filter params (interface=public, service_id=7b66073285e44f6e8b7df369412a52b8, region_id=RegionOne) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:34.389305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5e9be-4df7-47b0-9dad-d4fe738153f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:34.390979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5e9be-4df7-47b0-9dad-d4fe738153f0 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:34.403615 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 379/771] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 18:43:34 2022] GET /identity/v3/endpoints?service_id=7b66073285e44f6e8b7df369412a52b8&interface=public®ion_id=RegionOne => generated 194 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:34.409626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5e9be-4df7-47b0-9dad-d4fe738153f0 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:34.470661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 393/772] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:34 2022] PUT /identity/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/e628cb75e0c74f27a390945c8a04335c/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 18:43:34.568742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:34.588764 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5e9be-4df7-47b0-9dad-d4fe738153f0 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:34.644356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050558c0-d790-48b8-8d14-8473f4f7a4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:34.645146 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050558c0-d790-48b8-8d14-8473f4f7a4d5 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.645332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050558c0-d790-48b8-8d14-8473f4f7a4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.645611 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050558c0-d790-48b8-8d14-8473f4f7a4d5 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.646301 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050558c0-d790-48b8-8d14-8473f4f7a4d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:34.647718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050558c0-d790-48b8-8d14-8473f4f7a4d5 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:34.660447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 380/773] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 18:43:34 2022] GET /identity/v3/services => generated 772 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:34.766200 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb9f7c-f896-48cc-b05b-07b51ddd329a None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.766973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb9f7c-f896-48cc-b05b-07b51ddd329a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.766973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb9f7c-f896-48cc-b05b-07b51ddd329a None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.768410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 394/774] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:34.776372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc171d-beef-4d98-9ab3-f50b2d809479 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:34.777069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc171d-beef-4d98-9ab3-f50b2d809479 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:34.777605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc171d-beef-4d98-9ab3-f50b2d809479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:34.812159 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-26cc171d-beef-4d98-9ab3-f50b2d809479 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:34.813844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-26cc171d-beef-4d98-9ab3-f50b2d809479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:34.911808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 381/775] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:34.936283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb9f7c-f896-48cc-b05b-07b51ddd329a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:34.983740 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:35.095115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb9f7c-f896-48cc-b05b-07b51ddd329a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:35.142302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9ef23-c214-421f-b8a7-8cd6ea3cd4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:35.143156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9ef23-c214-421f-b8a7-8cd6ea3cd4b1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:35.143444 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9ef23-c214-421f-b8a7-8cd6ea3cd4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:35.143722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9ef23-c214-421f-b8a7-8cd6ea3cd4b1 None admin] PATH_INFO: `/v3/services/network` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:35.144392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9ef23-c214-421f-b8a7-8cd6ea3cd4b1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:35.145815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9ef23-c214-421f-b8a7-8cd6ea3cd4b1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-01f9ef23-c214-421f-b8a7-8cd6ea3cd4b1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:35.172402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:35.173494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:35.174470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 395/776] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:34 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:35.183649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc171d-beef-4d98-9ab3-f50b2d809479 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:35.354967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:35.381627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc171d-beef-4d98-9ab3-f50b2d809479 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:35.440843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c365f3-1a8c-43a7-9542-7fc36a944b9e None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:35.441165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c365f3-1a8c-43a7-9542-7fc36a944b9e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:35.441410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c365f3-1a8c-43a7-9542-7fc36a944b9e None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:35.443151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 396/777] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:35.449071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc63dcb-1b2e-4d60-afdb-98bd52850bb8 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:35.449362 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc63dcb-1b2e-4d60-afdb-98bd52850bb8 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:35.449641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc63dcb-1b2e-4d60-afdb-98bd52850bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:35.584465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-5bc63dcb-1b2e-4d60-afdb-98bd52850bb8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:35.587049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5bc63dcb-1b2e-4d60-afdb-98bd52850bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:35.641556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7a709-6085-45b8-9398-0e5f694d8dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:35.642248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7a709-6085-45b8-9398-0e5f694d8dc2 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:35.642565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7a709-6085-45b8-9398-0e5f694d8dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:35.642847 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7a709-6085-45b8-9398-0e5f694d8dc2 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:35.643574 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7a709-6085-45b8-9398-0e5f694d8dc2 None admin] RBAC: Adding query filter params (name=network) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:35.643891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7a709-6085-45b8-9398-0e5f694d8dc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:35.645435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7a709-6085-45b8-9398-0e5f694d8dc2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:35.656979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 382/778] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 474 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:35.662974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7a709-6085-45b8-9398-0e5f694d8dc2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:35.741229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 397/779] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:35.787474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc63dcb-1b2e-4d60-afdb-98bd52850bb8 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:35.846026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:35.864015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7a709-6085-45b8-9398-0e5f694d8dc2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:35.912448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f0328-e271-4bf2-a696-4b7bd941849e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:35.913510 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f0328-e271-4bf2-a696-4b7bd941849e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:35.914552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f0328-e271-4bf2-a696-4b7bd941849e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:35.915224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f0328-e271-4bf2-a696-4b7bd941849e None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:35.916254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f0328-e271-4bf2-a696-4b7bd941849e None admin] RBAC: Adding query filter params (type=network) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:35.916577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f0328-e271-4bf2-a696-4b7bd941849e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:35.918108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f0328-e271-4bf2-a696-4b7bd941849e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:35.926244 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 383/780] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:35.934840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f0328-e271-4bf2-a696-4b7bd941849e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:35.956551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:36.013868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc63dcb-1b2e-4d60-afdb-98bd52850bb8 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:36.078149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e0d1d-8979-4d82-9301-b19d4ea52587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:36.079342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e0d1d-8979-4d82-9301-b19d4ea52587 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.079848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e0d1d-8979-4d82-9301-b19d4ea52587 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.080505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e0d1d-8979-4d82-9301-b19d4ea52587 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.096414 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e0d1d-8979-4d82-9301-b19d4ea52587 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-f27e0d1d-8979-4d82-9301-b19d4ea52587 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:36.108462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:36.109868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:36.111307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:36.111307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:36.111307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:36.111307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:36.111307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:36.111307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:36.111307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:36.111307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:36.111307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:36.111307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:36.111307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:36.111307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:36.111307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 398/781] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:36.122454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e0d1d-8979-4d82-9301-b19d4ea52587 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:36.142600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f0328-e271-4bf2-a696-4b7bd941849e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:36.218199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4d0eb-6b5f-4aa2-b00d-0127b661a43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:36.218926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4d0eb-6b5f-4aa2-b00d-0127b661a43a None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.219213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4d0eb-6b5f-4aa2-b00d-0127b661a43a None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.219493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4d0eb-6b5f-4aa2-b00d-0127b661a43a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.220298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4d0eb-6b5f-4aa2-b00d-0127b661a43a None admin] RBAC: Adding query filter params (interface=public, service_id=bd0e6467cbc2483ea5420979a5077fac, region_id=RegionOne) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:36.220620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4d0eb-6b5f-4aa2-b00d-0127b661a43a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:36.222018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4d0eb-6b5f-4aa2-b00d-0127b661a43a None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:36.236641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 384/782] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 18:43:35 2022] GET /identity/v3/endpoints?service_id=bd0e6467cbc2483ea5420979a5077fac&interface=public®ion_id=RegionOne => generated 194 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:36.242454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4d0eb-6b5f-4aa2-b00d-0127b661a43a None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:36.289559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:36.305391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e0d1d-8979-4d82-9301-b19d4ea52587 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:36.371532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71643cb0-b715-42cb-9e34-fd5410d9eafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:36.372307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71643cb0-b715-42cb-9e34-fd5410d9eafe None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.372601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71643cb0-b715-42cb-9e34-fd5410d9eafe None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.372884 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71643cb0-b715-42cb-9e34-fd5410d9eafe None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.373900 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71643cb0-b715-42cb-9e34-fd5410d9eafe None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:36.374262 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71643cb0-b715-42cb-9e34-fd5410d9eafe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:36.375733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71643cb0-b715-42cb-9e34-fd5410d9eafe None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:36.390430 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:36.401319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 399/783] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:36.408411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71643cb0-b715-42cb-9e34-fd5410d9eafe None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:36.450039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4d0eb-6b5f-4aa2-b00d-0127b661a43a None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:36.552568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffed104-b2be-4512-bbb2-64f72a4c814b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:36.553187 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffed104-b2be-4512-bbb2-64f72a4c814b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.553481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffed104-b2be-4512-bbb2-64f72a4c814b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.553763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffed104-b2be-4512-bbb2-64f72a4c814b None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.555079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffed104-b2be-4512-bbb2-64f72a4c814b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:36.556476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffed104-b2be-4512-bbb2-64f72a4c814b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:36.577423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 385/784] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 18:43:36 2022] GET /identity/v3/services => generated 772 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:36.582477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71643cb0-b715-42cb-9e34-fd5410d9eafe None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:36.613335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83afa2-c0ab-476c-9ba5-96bfbfc6a994 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.613722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83afa2-c0ab-476c-9ba5-96bfbfc6a994 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.613951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83afa2-c0ab-476c-9ba5-96bfbfc6a994 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.615657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 386/785] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:36.629310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccc19f-fbcf-42b9-91c2-56d1803c3f63 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.629838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccc19f-fbcf-42b9-91c2-56d1803c3f63 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.630368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccc19f-fbcf-42b9-91c2-56d1803c3f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.656388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66a73c-5335-404d-afa5-457e8f205d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:36.657604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66a73c-5335-404d-afa5-457e8f205d28 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:36.657894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66a73c-5335-404d-afa5-457e8f205d28 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:36.658178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66a73c-5335-404d-afa5-457e8f205d28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:36.684023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-a1ccc19f-fbcf-42b9-91c2-56d1803c3f63 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:36.686013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a1ccc19f-fbcf-42b9-91c2-56d1803c3f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:36.691382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66a73c-5335-404d-afa5-457e8f205d28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:36.692994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66a73c-5335-404d-afa5-457e8f205d28 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-cf66a73c-5335-404d-afa5-457e8f205d28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:36.734834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:36.736959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:36.739093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:36.739093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:36.739093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:36.739093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:36.739093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 400/786] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:36.747711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66a73c-5335-404d-afa5-457e8f205d28 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:36.755089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:36.843954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 387/787] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:43:36.886392 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccc19f-fbcf-42b9-91c2-56d1803c3f63 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:36.965060 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66a73c-5335-404d-afa5-457e8f205d28 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.025882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.050166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3759cd-023c-4aea-8ddf-411cbc66cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.051232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3759cd-023c-4aea-8ddf-411cbc66cf0d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.051598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3759cd-023c-4aea-8ddf-411cbc66cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.052054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3759cd-023c-4aea-8ddf-411cbc66cf0d None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.053095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3759cd-023c-4aea-8ddf-411cbc66cf0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:37.053616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3759cd-023c-4aea-8ddf-411cbc66cf0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:37.055639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3759cd-023c-4aea-8ddf-411cbc66cf0d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:37.088331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccc19f-fbcf-42b9-91c2-56d1803c3f63 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.093142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 401/788] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:37.099631 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3759cd-023c-4aea-8ddf-411cbc66cf0d None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.165061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751c851-2a43-4db0-90f8-c3536f2b829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.166021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751c851-2a43-4db0-90f8-c3536f2b829a None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.166319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751c851-2a43-4db0-90f8-c3536f2b829a None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.166619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751c851-2a43-4db0-90f8-c3536f2b829a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.194881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751c851-2a43-4db0-90f8-c3536f2b829a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-b751c851-2a43-4db0-90f8-c3536f2b829a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:37.204296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:37.205447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:37.206734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:37.206734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:37.206734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:37.206734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:37.206734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:37.206734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:37.206734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:37.206734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:37.206734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:37.206734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:37.206734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:37.206734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:37.206734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 388/789] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:37.216471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751c851-2a43-4db0-90f8-c3536f2b829a None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.287330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.361224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3759cd-023c-4aea-8ddf-411cbc66cf0d None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.407167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03639e8a-b279-4f36-94f0-a9d1237c5944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.408012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03639e8a-b279-4f36-94f0-a9d1237c5944 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.408297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03639e8a-b279-4f36-94f0-a9d1237c5944 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.408576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03639e8a-b279-4f36-94f0-a9d1237c5944 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.419686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03639e8a-b279-4f36-94f0-a9d1237c5944 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:37.421186 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03639e8a-b279-4f36-94f0-a9d1237c5944 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:37.435564 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751c851-2a43-4db0-90f8-c3536f2b829a None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-03639e8a-b279-4f36-94f0-a9d1237c5944 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:37.440329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:37.441361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:37.442749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:37.442749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:37.442749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:37.442749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:37.442749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:37.442749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:37.442749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:37.442749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:37.442749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:37.442749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:37.442749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 402/790] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:37.443320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da6f91-dd39-40a5-9610-b25209a65d27 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.443618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da6f91-dd39-40a5-9610-b25209a65d27 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.444775 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da6f91-dd39-40a5-9610-b25209a65d27 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.446409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 403/791] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:37.456343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec0d5b-1e59-4a0b-8e08-b44854442514 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.457128 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec0d5b-1e59-4a0b-8e08-b44854442514 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.457673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec0d5b-1e59-4a0b-8e08-b44854442514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.480922 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.484989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-c1ec0d5b-1e59-4a0b-8e08-b44854442514 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:37.487123 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c1ec0d5b-1e59-4a0b-8e08-b44854442514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:37.493899 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85803e07-62c6-4cdd-8904-d45a120781ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.495095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85803e07-62c6-4cdd-8904-d45a120781ef None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.497556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85803e07-62c6-4cdd-8904-d45a120781ef None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.497556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85803e07-62c6-4cdd-8904-d45a120781ef None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.497556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85803e07-62c6-4cdd-8904-d45a120781ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:37.497556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85803e07-62c6-4cdd-8904-d45a120781ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:37.498674 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85803e07-62c6-4cdd-8904-d45a120781ef None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:37.512872 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 389/792] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:37.513857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85803e07-62c6-4cdd-8904-d45a120781ef None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.610495 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 404/793] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:37.612734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09ff9e-1bef-4cf2-89d0-f48465f4a588 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.613378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09ff9e-1bef-4cf2-89d0-f48465f4a588 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.614912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09ff9e-1bef-4cf2-89d0-f48465f4a588 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.616731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 405/794] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:37.619147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09ff9e-1bef-4cf2-89d0-f48465f4a588 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.681189 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85803e07-62c6-4cdd-8904-d45a120781ef None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.699152 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.723309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec56881-60a2-4d25-808c-f7da539c714e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.724121 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec56881-60a2-4d25-808c-f7da539c714e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.724865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec56881-60a2-4d25-808c-f7da539c714e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.724865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec56881-60a2-4d25-808c-f7da539c714e None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.725872 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec56881-60a2-4d25-808c-f7da539c714e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:37.726205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec56881-60a2-4d25-808c-f7da539c714e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:37.728960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec56881-60a2-4d25-808c-f7da539c714e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:37.750833 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 390/795] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:37.756128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a959d-d838-4319-ba3b-8c06a32550f2 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.756809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a959d-d838-4319-ba3b-8c06a32550f2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.757102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a959d-d838-4319-ba3b-8c06a32550f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.760742 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09ff9e-1bef-4cf2-89d0-f48465f4a588 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.782199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-732a959d-d838-4319-ba3b-8c06a32550f2 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:37.783600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-732a959d-d838-4319-ba3b-8c06a32550f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:37.809807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a06be-98b9-4547-86a0-43773b125ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:37.810827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a06be-98b9-4547-86a0-43773b125ebe None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:37.811114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a06be-98b9-4547-86a0-43773b125ebe None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:37.811395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a06be-98b9-4547-86a0-43773b125ebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:37.826647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a06be-98b9-4547-86a0-43773b125ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:37.828288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a06be-98b9-4547-86a0-43773b125ebe None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:37.840945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-082a06be-98b9-4547-86a0-43773b125ebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:37.850922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:37.852722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:37.853785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:37.853785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:37.853785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:37.853785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:37.853785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 406/796] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:37.853785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a06be-98b9-4547-86a0-43773b125ebe None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.881320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 391/797] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:37.881320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a959d-d838-4319-ba3b-8c06a32550f2 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:37.980272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a06be-98b9-4547-86a0-43773b125ebe None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:37.988475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:37.990744 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a959d-d838-4319-ba3b-8c06a32550f2 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.025907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb9ea3-9ce2-498e-a00b-1cca4444e6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.027360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb9ea3-9ce2-498e-a00b-1cca4444e6ac None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.027912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb9ea3-9ce2-498e-a00b-1cca4444e6ac None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.028377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb9ea3-9ce2-498e-a00b-1cca4444e6ac None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.029381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb9ea3-9ce2-498e-a00b-1cca4444e6ac None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.031005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb9ea3-9ce2-498e-a00b-1cca4444e6ac None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.031135 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce7cf0-0570-45e7-8474-055bafa59d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.031935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce7cf0-0570-45e7-8474-055bafa59d16 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.032230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce7cf0-0570-45e7-8474-055bafa59d16 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.032510 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce7cf0-0570-45e7-8474-055bafa59d16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.044089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce7cf0-0570-45e7-8474-055bafa59d16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.045551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce7cf0-0570-45e7-8474-055bafa59d16 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-7efb9ea3-9ce2-498e-a00b-1cca4444e6ac None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:38.047879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:38.048851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 18:43:38.049670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:38.058850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 407/798] 10.222.0.17 () {56 vars in 1129 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:38.058850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb9ea3-9ce2-498e-a00b-1cca4444e6ac None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-10ce7cf0-0570-45e7-8474-055bafa59d16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:38.078154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:38.079960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:38.088242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:38.088242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:38.088242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:38.088242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:38.088242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 392/799] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:38.088242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce7cf0-0570-45e7-8474-055bafa59d16 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.122767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.187746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb9ea3-9ce2-498e-a00b-1cca4444e6ac None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.218134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce7cf0-0570-45e7-8474-055bafa59d16 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.230890 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983121e-ba86-4e34-9762-9aea4abc6416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.231653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983121e-ba86-4e34-9762-9aea4abc6416 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.231956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983121e-ba86-4e34-9762-9aea4abc6416 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.232245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983121e-ba86-4e34-9762-9aea4abc6416 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.232972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983121e-ba86-4e34-9762-9aea4abc6416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:38.233324 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983121e-ba86-4e34-9762-9aea4abc6416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.234807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983121e-ba86-4e34-9762-9aea4abc6416 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.253923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 408/800] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:38.255342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983121e-ba86-4e34-9762-9aea4abc6416 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.258117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22233dd8-496f-43d0-ac1a-67bb8f9de178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.259567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22233dd8-496f-43d0-ac1a-67bb8f9de178 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.260069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22233dd8-496f-43d0-ac1a-67bb8f9de178 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.261268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22233dd8-496f-43d0-ac1a-67bb8f9de178 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.274220 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22233dd8-496f-43d0-ac1a-67bb8f9de178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-22233dd8-496f-43d0-ac1a-67bb8f9de178 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:38.282557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:38.283540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:38.284854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:38.284854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:38.284854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:38.284854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:38.284854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:38.284854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:38.284854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:38.284854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:38.284854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:38.284854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:38.284854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:38.284854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:38.284854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 393/801] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:38.285809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22233dd8-496f-43d0-ac1a-67bb8f9de178 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.302178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.381490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983121e-ba86-4e34-9762-9aea4abc6416 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.390452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.402302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22233dd8-496f-43d0-ac1a-67bb8f9de178 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.421163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774b7f4-365e-4ef2-99e1-438838756412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.421872 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774b7f4-365e-4ef2-99e1-438838756412 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.422166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774b7f4-365e-4ef2-99e1-438838756412 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.422526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774b7f4-365e-4ef2-99e1-438838756412 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.423428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774b7f4-365e-4ef2-99e1-438838756412 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:38.423745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774b7f4-365e-4ef2-99e1-438838756412 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.425460 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774b7f4-365e-4ef2-99e1-438838756412 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.430368 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 409/802] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:38.431473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774b7f4-365e-4ef2-99e1-438838756412 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.449685 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1094d-9b47-4c03-8ab0-19dfa4686bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.450523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1094d-9b47-4c03-8ab0-19dfa4686bf0 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.450891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1094d-9b47-4c03-8ab0-19dfa4686bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.451181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1094d-9b47-4c03-8ab0-19dfa4686bf0 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.451917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1094d-9b47-4c03-8ab0-19dfa4686bf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:38.452238 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1094d-9b47-4c03-8ab0-19dfa4686bf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.453626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1094d-9b47-4c03-8ab0-19dfa4686bf0 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.467111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 394/803] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:38.468187 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1094d-9b47-4c03-8ab0-19dfa4686bf0 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.557355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.566662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774b7f4-365e-4ef2-99e1-438838756412 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.583355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1094d-9b47-4c03-8ab0-19dfa4686bf0 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.610055 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662dbff-e1f2-40d8-97ea-e9d46f23ec7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.610916 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662dbff-e1f2-40d8-97ea-e9d46f23ec7b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.611207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662dbff-e1f2-40d8-97ea-e9d46f23ec7b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.611480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662dbff-e1f2-40d8-97ea-e9d46f23ec7b None admin] PATH_INFO: `/v3/users/swift` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.621655 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662dbff-e1f2-40d8-97ea-e9d46f23ec7b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.623417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662dbff-e1f2-40d8-97ea-e9d46f23ec7b None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.624740 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c5fd8-90a2-4983-8fee-e6fca1c119b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.625540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c5fd8-90a2-4983-8fee-e6fca1c119b6 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.626004 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c5fd8-90a2-4983-8fee-e6fca1c119b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.626259 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c5fd8-90a2-4983-8fee-e6fca1c119b6 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.627020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c5fd8-90a2-4983-8fee-e6fca1c119b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:38.627338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c5fd8-90a2-4983-8fee-e6fca1c119b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.628929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c5fd8-90a2-4983-8fee-e6fca1c119b6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.643702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 395/804] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:38.643702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c5fd8-90a2-4983-8fee-e6fca1c119b6 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-2662dbff-e1f2-40d8-97ea-e9d46f23ec7b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:38.646615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:38.647556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:38.649418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:38.649418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:38.649418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:38.649418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:38.649418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:38.649418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:38.649418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:38.649418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:38.649418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:38.649418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:38.649418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 410/805] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:38.655179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662dbff-e1f2-40d8-97ea-e9d46f23ec7b None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.668202 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.765876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c5fd8-90a2-4983-8fee-e6fca1c119b6 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.771844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662dbff-e1f2-40d8-97ea-e9d46f23ec7b None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.805077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c49555-3f78-4e11-9a4b-d87558e38bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.806101 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c49555-3f78-4e11-9a4b-d87558e38bcf None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.806631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c49555-3f78-4e11-9a4b-d87558e38bcf None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.806906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c49555-3f78-4e11-9a4b-d87558e38bcf None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.807570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c49555-3f78-4e11-9a4b-d87558e38bcf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:38.807882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c49555-3f78-4e11-9a4b-d87558e38bcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.809423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c49555-3f78-4e11-9a4b-d87558e38bcf None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.812207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248d576-c3d6-4c9c-918e-d032c39a16c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.813024 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248d576-c3d6-4c9c-918e-d032c39a16c8 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.813356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248d576-c3d6-4c9c-918e-d032c39a16c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.813589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248d576-c3d6-4c9c-918e-d032c39a16c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.820011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 396/806] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:38.820011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c49555-3f78-4e11-9a4b-d87558e38bcf None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.828590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248d576-c3d6-4c9c-918e-d032c39a16c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.829804 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248d576-c3d6-4c9c-918e-d032c39a16c8 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-e248d576-c3d6-4c9c-918e-d032c39a16c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:38.845232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:38.846280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:38.847194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:38.847194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:38.847194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:38.847194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:38.847194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:38.847194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:38.847194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:38.847194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:38.847194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 411/807] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:38.847194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248d576-c3d6-4c9c-918e-d032c39a16c8 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:38.868849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.909357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:38.937939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c49555-3f78-4e11-9a4b-d87558e38bcf None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.970878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248d576-c3d6-4c9c-918e-d032c39a16c8 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:38.975762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1f750-6e77-4096-b96a-6107b9c6c8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:38.976779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1f750-6e77-4096-b96a-6107b9c6c8b5 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:38.977078 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1f750-6e77-4096-b96a-6107b9c6c8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:38.977362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1f750-6e77-4096-b96a-6107b9c6c8b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:38.988058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1f750-6e77-4096-b96a-6107b9c6c8b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:38.989717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1f750-6e77-4096-b96a-6107b9c6c8b5 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.009266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a06b3e-bef7-4498-bc93-9aa188b68bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.009947 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a06b3e-bef7-4498-bc93-9aa188b68bff None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-3dc1f750-6e77-4096-b96a-6107b9c6c8b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:39.010069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:39.011096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:39.012174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:39.012174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:39.012174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:39.012174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:39.012533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a06b3e-bef7-4498-bc93-9aa188b68bff None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.012533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a06b3e-bef7-4498-bc93-9aa188b68bff None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.012533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a06b3e-bef7-4498-bc93-9aa188b68bff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:39.012533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a06b3e-bef7-4498-bc93-9aa188b68bff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.013092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a06b3e-bef7-4498-bc93-9aa188b68bff None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.014732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 397/808] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:39.015859 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1f750-6e77-4096-b96a-6107b9c6c8b5 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.037421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 412/809] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Jul 25 18:43:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:39.038610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a06b3e-bef7-4498-bc93-9aa188b68bff None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.112996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:39.131096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1f750-6e77-4096-b96a-6107b9c6c8b5 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.144878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a06b3e-bef7-4498-bc93-9aa188b68bff None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.165130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:39.173114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b052d05-ea71-47a9-8499-3eebccc925df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.173782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b052d05-ea71-47a9-8499-3eebccc925df None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.177595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b052d05-ea71-47a9-8499-3eebccc925df None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.177595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b052d05-ea71-47a9-8499-3eebccc925df None admin] PATH_INFO: `/v3/endpoints` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.177595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b052d05-ea71-47a9-8499-3eebccc925df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.177595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b052d05-ea71-47a9-8499-3eebccc925df None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.181144 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8065b87-b295-4be3-a44e-88a3a0e191c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.181818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8065b87-b295-4be3-a44e-88a3a0e191c8 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.182104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8065b87-b295-4be3-a44e-88a3a0e191c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.182415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8065b87-b295-4be3-a44e-88a3a0e191c8 None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.183205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8065b87-b295-4be3-a44e-88a3a0e191c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:39.183512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8065b87-b295-4be3-a44e-88a3a0e191c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.184942 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8065b87-b295-4be3-a44e-88a3a0e191c8 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.198289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 413/810] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:39.199134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8065b87-b295-4be3-a44e-88a3a0e191c8 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.284835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 398/811] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 18:43:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:39.285804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b052d05-ea71-47a9-8499-3eebccc925df None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.307343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8065b87-b295-4be3-a44e-88a3a0e191c8 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.354237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:39.366976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c146d-ce2a-4523-82f6-4ec42250825a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.368570 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c146d-ce2a-4523-82f6-4ec42250825a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.368867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c146d-ce2a-4523-82f6-4ec42250825a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.369135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c146d-ce2a-4523-82f6-4ec42250825a None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.369893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c146d-ce2a-4523-82f6-4ec42250825a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:39.370260 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c146d-ce2a-4523-82f6-4ec42250825a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.371714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c146d-ce2a-4523-82f6-4ec42250825a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.386883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 414/812] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:39.388299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422252e-591a-4060-8352-4feb7466389a None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.388661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422252e-591a-4060-8352-4feb7466389a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.388972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422252e-591a-4060-8352-4feb7466389a None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.394542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 415/813] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:39.394542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422252e-591a-4060-8352-4feb7466389a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.427257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b052d05-ea71-47a9-8499-3eebccc925df None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.467734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffadbb-6b77-49e4-90ff-5bd642fed63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.468726 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffadbb-6b77-49e4-90ff-5bd642fed63a None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.469017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffadbb-6b77-49e4-90ff-5bd642fed63a None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.469301 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffadbb-6b77-49e4-90ff-5bd642fed63a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.483993 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffadbb-6b77-49e4-90ff-5bd642fed63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.486613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffadbb-6b77-49e4-90ff-5bd642fed63a None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-59ffadbb-6b77-49e4-90ff-5bd642fed63a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:39.509150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:39.510083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:39.511013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:39.511013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:39.511013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:39.511013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:39.511013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 399/814] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:39.511301 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffadbb-6b77-49e4-90ff-5bd642fed63a None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.532653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422252e-591a-4060-8352-4feb7466389a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.554128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:39.574553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f38560-74d4-46f3-961a-8e87ceb40ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.575399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f38560-74d4-46f3-961a-8e87ceb40ee9 None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.575684 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f38560-74d4-46f3-961a-8e87ceb40ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.575978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f38560-74d4-46f3-961a-8e87ceb40ee9 None admin] PATH_INFO: `/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/c6a88af6e45549f1ab00036923f6ba04/roles/1f308d13b8d04c0b9309a0916dee061f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.593149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f38560-74d4-46f3-961a-8e87ceb40ee9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d7c3b139f284435aea4ff08a2f0952b, user_id=c6a88af6e45549f1ab00036923f6ba04, role_id=1f308d13b8d04c0b9309a0916dee061f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.594827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f38560-74d4-46f3-961a-8e87ceb40ee9 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.631698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffadbb-6b77-49e4-90ff-5bd642fed63a None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.668806 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845885fd-1098-483b-9909-fa6c55905b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.669616 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845885fd-1098-483b-9909-fa6c55905b90 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.671355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845885fd-1098-483b-9909-fa6c55905b90 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.671355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845885fd-1098-483b-9909-fa6c55905b90 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.678387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845885fd-1098-483b-9909-fa6c55905b90 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.679820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845885fd-1098-483b-9909-fa6c55905b90 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-845885fd-1098-483b-9909-fa6c55905b90 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:39.695944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:39.696945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:39.697955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:39.697955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:39.697955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:39.697955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:39.697955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:39.697955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:39.697955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:39.697955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:39.697955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:43:39.697955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:39.697955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 400/815] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:39.698682 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40487ee-8601-46c4-bfb7-79196c873fed None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.699387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40487ee-8601-46c4-bfb7-79196c873fed None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.699387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40487ee-8601-46c4-bfb7-79196c873fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.710666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 416/816] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:39 2022] PUT /identity/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/c6a88af6e45549f1ab00036923f6ba04/roles/1f308d13b8d04c0b9309a0916dee061f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:39.710666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f38560-74d4-46f3-961a-8e87ceb40ee9 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.719873 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-a40487ee-8601-46c4-bfb7-79196c873fed None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:39.721143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a40487ee-8601-46c4-bfb7-79196c873fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:39.770412 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:39.805154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 401/817] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:39.806281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40487ee-8601-46c4-bfb7-79196c873fed None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:39.842845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:39.889557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f38560-74d4-46f3-961a-8e87ceb40ee9 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.943930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40602fcc-8b3e-4c07-8013-014cf9a5bb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:39.945017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40602fcc-8b3e-4c07-8013-014cf9a5bb2c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:39.945944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40602fcc-8b3e-4c07-8013-014cf9a5bb2c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:39.946525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40602fcc-8b3e-4c07-8013-014cf9a5bb2c None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:39.947329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40602fcc-8b3e-4c07-8013-014cf9a5bb2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:39.947691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40602fcc-8b3e-4c07-8013-014cf9a5bb2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:39.949108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40602fcc-8b3e-4c07-8013-014cf9a5bb2c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:39.952012 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40487ee-8601-46c4-bfb7-79196c873fed None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:39.966368 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 417/818] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:39.966368 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40602fcc-8b3e-4c07-8013-014cf9a5bb2c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.013651 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab509f82-ba76-4bfc-a404-8bf6fcaf1bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.014751 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab509f82-ba76-4bfc-a404-8bf6fcaf1bf4 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.015238 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab509f82-ba76-4bfc-a404-8bf6fcaf1bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.015793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab509f82-ba76-4bfc-a404-8bf6fcaf1bf4 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.016866 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab509f82-ba76-4bfc-a404-8bf6fcaf1bf4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:40.017394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab509f82-ba76-4bfc-a404-8bf6fcaf1bf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.019110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab509f82-ba76-4bfc-a404-8bf6fcaf1bf4 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.046698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 402/819] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:40.048416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab509f82-ba76-4bfc-a404-8bf6fcaf1bf4 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.096805 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.105363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40602fcc-8b3e-4c07-8013-014cf9a5bb2c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.149223 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12e13c-0eda-46d4-892d-43473baa403c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.150054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12e13c-0eda-46d4-892d-43473baa403c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.150351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12e13c-0eda-46d4-892d-43473baa403c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.150653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12e13c-0eda-46d4-892d-43473baa403c None admin] PATH_INFO: `/v3/services/network` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.151321 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12e13c-0eda-46d4-892d-43473baa403c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.152863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12e13c-0eda-46d4-892d-43473baa403c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-fa12e13c-0eda-46d4-892d-43473baa403c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:40.169837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:40.171030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 18:43:40.171948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:40.176175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 418/820] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:39 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:40.177482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12e13c-0eda-46d4-892d-43473baa403c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.191118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.217133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab509f82-ba76-4bfc-a404-8bf6fcaf1bf4 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.279865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33872a-4b4d-4393-ad7b-c05bfb3d1221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.281249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33872a-4b4d-4393-ad7b-c05bfb3d1221 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.282013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33872a-4b4d-4393-ad7b-c05bfb3d1221 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.282504 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33872a-4b4d-4393-ad7b-c05bfb3d1221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.295962 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33872a-4b4d-4393-ad7b-c05bfb3d1221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.297766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33872a-4b4d-4393-ad7b-c05bfb3d1221 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.305309 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12e13c-0eda-46d4-892d-43473baa403c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-bb33872a-4b4d-4393-ad7b-c05bfb3d1221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:40.319130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:40.320406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:40.321752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:40.321752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:40.321752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:40.321752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:40.321752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:40.321752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:40.321752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:40.321752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:40.325837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 403/821] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:40.326635 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33872a-4b4d-4393-ad7b-c05bfb3d1221 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.374594 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3763ec-737e-4599-a403-5adbefa55488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.375606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3763ec-737e-4599-a403-5adbefa55488 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.376095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3763ec-737e-4599-a403-5adbefa55488 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.376599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3763ec-737e-4599-a403-5adbefa55488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.418372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.470256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3763ec-737e-4599-a403-5adbefa55488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.471924 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3763ec-737e-4599-a403-5adbefa55488 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.478892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33872a-4b4d-4393-ad7b-c05bfb3d1221 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.530858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e858f-0dfd-49f5-a7fd-b756ded59141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.532327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e858f-0dfd-49f5-a7fd-b756ded59141 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.532862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e858f-0dfd-49f5-a7fd-b756ded59141 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.533346 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e858f-0dfd-49f5-a7fd-b756ded59141 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.534287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e858f-0dfd-49f5-a7fd-b756ded59141 None admin] RBAC: Adding query filter params (name=network) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:40.535403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e858f-0dfd-49f5-a7fd-b756ded59141 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.537301 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e858f-0dfd-49f5-a7fd-b756ded59141 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.545708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 404/822] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:40.547148 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e858f-0dfd-49f5-a7fd-b756ded59141 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.565691 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.598583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 419/823] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:43:40.600535 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3763ec-737e-4599-a403-5adbefa55488 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.677881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e858f-0dfd-49f5-a7fd-b756ded59141 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.683745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.728664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3763ec-737e-4599-a403-5adbefa55488 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.732673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a3f16-83c9-42d1-a186-7f02344885fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.734219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a3f16-83c9-42d1-a186-7f02344885fb None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.735170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a3f16-83c9-42d1-a186-7f02344885fb None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.735999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a3f16-83c9-42d1-a186-7f02344885fb None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.737564 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a3f16-83c9-42d1-a186-7f02344885fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:40.738692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a3f16-83c9-42d1-a186-7f02344885fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.740855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a3f16-83c9-42d1-a186-7f02344885fb None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.759019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 405/824] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:40.759838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a3f16-83c9-42d1-a186-7f02344885fb None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.780926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61737e-2e48-4ce6-bb2f-d069f08e2d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.782291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61737e-2e48-4ce6-bb2f-d069f08e2d33 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.782823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61737e-2e48-4ce6-bb2f-d069f08e2d33 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.783703 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61737e-2e48-4ce6-bb2f-d069f08e2d33 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.784683 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61737e-2e48-4ce6-bb2f-d069f08e2d33 None admin] RBAC: Adding query filter params (type=network) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:40.785205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61737e-2e48-4ce6-bb2f-d069f08e2d33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.787564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61737e-2e48-4ce6-bb2f-d069f08e2d33 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.798191 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 420/825] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:40.799684 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61737e-2e48-4ce6-bb2f-d069f08e2d33 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:40.848080 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:40.891889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a3f16-83c9-42d1-a186-7f02344885fb None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.933327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f173d-7d91-4622-bc46-bf12b656ef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:40.934447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f173d-7d91-4622-bc46-bf12b656ef1d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:40.934749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f173d-7d91-4622-bc46-bf12b656ef1d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:40.935181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f173d-7d91-4622-bc46-bf12b656ef1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:40.948338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f173d-7d91-4622-bc46-bf12b656ef1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:40.949761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f173d-7d91-4622-bc46-bf12b656ef1d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:40.968044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61737e-2e48-4ce6-bb2f-d069f08e2d33 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-1b0f173d-7d91-4622-bc46-bf12b656ef1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:40.968748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:40.969613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:40.970524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:40.970524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:40.970524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:40.970524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:40.970524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 406/826] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:40.971323 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f173d-7d91-4622-bc46-bf12b656ef1d None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.003588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:41.014949 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cfb7b-b647-43c3-ab76-ec3159efe51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.024923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cfb7b-b647-43c3-ab76-ec3159efe51c None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.024923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cfb7b-b647-43c3-ab76-ec3159efe51c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.024923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cfb7b-b647-43c3-ab76-ec3159efe51c None admin] PATH_INFO: `/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/1ee9152572224cc499229e0fa2b81ac5/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.024923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cfb7b-b647-43c3-ab76-ec3159efe51c None admin] RBAC: Authorizing `identity:create_grant(project_id=5d7c3b139f284435aea4ff08a2f0952b, user_id=1ee9152572224cc499229e0fa2b81ac5, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:41.024923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cfb7b-b647-43c3-ab76-ec3159efe51c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.087492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f173d-7d91-4622-bc46-bf12b656ef1d None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:41.109449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:41.130088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003963d-5583-4080-abd8-596e6b23b82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.130088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003963d-5583-4080-abd8-596e6b23b82d None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.130088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003963d-5583-4080-abd8-596e6b23b82d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.130088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003963d-5583-4080-abd8-596e6b23b82d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.130552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003963d-5583-4080-abd8-596e6b23b82d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:41.130552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003963d-5583-4080-abd8-596e6b23b82d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.131024 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 421/827] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:40 2022] PUT /identity/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/1ee9152572224cc499229e0fa2b81ac5/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:41.132747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cfb7b-b647-43c3-ab76-ec3159efe51c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.308516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 407/828] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 18:43:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:41.314266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cfb7b-b647-43c3-ab76-ec3159efe51c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:41.385760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438264ca-2420-4b2a-98c5-13a478d7c23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.386506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438264ca-2420-4b2a-98c5-13a478d7c23f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.386796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438264ca-2420-4b2a-98c5-13a478d7c23f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.387080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438264ca-2420-4b2a-98c5-13a478d7c23f None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.387895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438264ca-2420-4b2a-98c5-13a478d7c23f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:41.388211 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438264ca-2420-4b2a-98c5-13a478d7c23f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:41.389681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438264ca-2420-4b2a-98c5-13a478d7c23f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.405802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 422/829] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:41.411336 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003963d-5583-4080-abd8-596e6b23b82d None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.553820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e7422-3acc-41fa-9d69-1b26ae89b88b None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.554353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e7422-3acc-41fa-9d69-1b26ae89b88b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.555017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e7422-3acc-41fa-9d69-1b26ae89b88b None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.556887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 423/830] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:41.564046 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e493ad6-fa4d-4cca-8449-45e3658daa8a None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.564314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003963d-5583-4080-abd8-596e6b23b82d None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:41.564798 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e493ad6-fa4d-4cca-8449-45e3658daa8a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.565273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e493ad6-fa4d-4cca-8449-45e3658daa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.588469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-3e493ad6-fa4d-4cca-8449-45e3658daa8a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:41.590076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3e493ad6-fa4d-4cca-8449-45e3658daa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:41.613621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263104e6-39ce-4bed-bad4-1f28e1c85741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.615041 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263104e6-39ce-4bed-bad4-1f28e1c85741 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.615342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263104e6-39ce-4bed-bad4-1f28e1c85741 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.615628 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263104e6-39ce-4bed-bad4-1f28e1c85741 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.628605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263104e6-39ce-4bed-bad4-1f28e1c85741 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:41.631846 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263104e6-39ce-4bed-bad4-1f28e1c85741 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-263104e6-39ce-4bed-bad4-1f28e1c85741 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:41.652722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:41.654290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:41.656109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:41.656109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:41.656109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:41.656109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:41.656109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:41.656109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:41.656109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:41.656109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:41.658455 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 408/831] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:41.660447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263104e6-39ce-4bed-bad4-1f28e1c85741 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.666773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:41.710255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:41.724917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 424/832] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:41.750072 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e493ad6-fa4d-4cca-8449-45e3658daa8a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.822031 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263104e6-39ce-4bed-bad4-1f28e1c85741 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:41.876831 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c248caf-ed5d-45ba-8622-9107a9d58c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.877781 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c248caf-ed5d-45ba-8622-9107a9d58c5e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.878071 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c248caf-ed5d-45ba-8622-9107a9d58c5e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.878617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c248caf-ed5d-45ba-8622-9107a9d58c5e None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.879852 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c248caf-ed5d-45ba-8622-9107a9d58c5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:41.880441 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c248caf-ed5d-45ba-8622-9107a9d58c5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:41.881903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c248caf-ed5d-45ba-8622-9107a9d58c5e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.882262 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e493ad6-fa4d-4cca-8449-45e3658daa8a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:41.902429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 409/833] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:41.905743 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c248caf-ed5d-45ba-8622-9107a9d58c5e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.926237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463e898-6eeb-4632-8e5b-5c45e492a181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:41.927208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463e898-6eeb-4632-8e5b-5c45e492a181 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:41.927497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463e898-6eeb-4632-8e5b-5c45e492a181 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:41.927780 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463e898-6eeb-4632-8e5b-5c45e492a181 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:41.928445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463e898-6eeb-4632-8e5b-5c45e492a181 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:41.930527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463e898-6eeb-4632-8e5b-5c45e492a181 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-0463e898-6eeb-4632-8e5b-5c45e492a181 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:41.943355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:41.944378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:41.945207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 425/834] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:41.950853 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463e898-6eeb-4632-8e5b-5c45e492a181 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:41.988687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:42.018662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:42.025090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c248caf-ed5d-45ba-8622-9107a9d58c5e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.057506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463e898-6eeb-4632-8e5b-5c45e492a181 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.060812 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dda113-d221-48d9-8ffb-445fd8db1d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.061487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dda113-d221-48d9-8ffb-445fd8db1d0d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.061801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dda113-d221-48d9-8ffb-445fd8db1d0d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.062079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dda113-d221-48d9-8ffb-445fd8db1d0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.062946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dda113-d221-48d9-8ffb-445fd8db1d0d None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=e628cb75e0c74f27a390945c8a04335c) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:42.063268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dda113-d221-48d9-8ffb-445fd8db1d0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.064710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dda113-d221-48d9-8ffb-445fd8db1d0d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.073549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 410/835] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/role_assignments?user.id=e628cb75e0c74f27a390945c8a04335c&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:42.075096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41c424-14aa-423a-92f4-cee1682014b2 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.075458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41c424-14aa-423a-92f4-cee1682014b2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.075888 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41c424-14aa-423a-92f4-cee1682014b2 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.081533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 411/836] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:42.083436 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63bdbb-054a-4004-ae78-9c6c2dcee4a5 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.083980 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63bdbb-054a-4004-ae78-9c6c2dcee4a5 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.084425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63bdbb-054a-4004-ae78-9c6c2dcee4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.106778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1234be-9ae0-4864-8318-350fa59bff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.107753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1234be-9ae0-4864-8318-350fa59bff28 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.108214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1234be-9ae0-4864-8318-350fa59bff28 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.108606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1234be-9ae0-4864-8318-350fa59bff28 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.109520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1234be-9ae0-4864-8318-350fa59bff28 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:42.109943 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1234be-9ae0-4864-8318-350fa59bff28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.110474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ec63bdbb-054a-4004-ae78-9c6c2dcee4a5 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:42.111644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1234be-9ae0-4864-8318-350fa59bff28 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.112039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ec63bdbb-054a-4004-ae78-9c6c2dcee4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:42.118384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 426/837] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:42.123834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1234be-9ae0-4864-8318-350fa59bff28 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.210356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 412/838] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:42.235552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63bdbb-054a-4004-ae78-9c6c2dcee4a5 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.278029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:42.286838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1234be-9ae0-4864-8318-350fa59bff28 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.343108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fdc55-af0f-43ef-96eb-46e8b17c00fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.343108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fdc55-af0f-43ef-96eb-46e8b17c00fd None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.343417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fdc55-af0f-43ef-96eb-46e8b17c00fd None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.343973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fdc55-af0f-43ef-96eb-46e8b17c00fd None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.345056 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64fdc55-af0f-43ef-96eb-46e8b17c00fd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:42.345513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64fdc55-af0f-43ef-96eb-46e8b17c00fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.347752 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64fdc55-af0f-43ef-96eb-46e8b17c00fd None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.354492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 427/839] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:42.358849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:42.370557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63bdbb-054a-4004-ae78-9c6c2dcee4a5 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.415925 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470296e7-0951-47d8-bea1-1eb6c5b615de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.416781 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470296e7-0951-47d8-bea1-1eb6c5b615de None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.417071 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470296e7-0951-47d8-bea1-1eb6c5b615de None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.417354 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470296e7-0951-47d8-bea1-1eb6c5b615de None admin] PATH_INFO: `/v3/roles/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.429834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470296e7-0951-47d8-bea1-1eb6c5b615de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-470296e7-0951-47d8-bea1-1eb6c5b615de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:42.437994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:42.447331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:42.448205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:42.448205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:42.448205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:42.448205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:42.448205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:42.448205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:42.448205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:42.448205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:42.448205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:42.448205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:42.448205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:43:42.448205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:42.448205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 413/840] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:42.448899 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fdc55-af0f-43ef-96eb-46e8b17c00fd None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.582310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fdc55-af0f-43ef-96eb-46e8b17c00fd None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.629560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4797c4-cb11-49e5-aa56-596a4b18048c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.630173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4797c4-cb11-49e5-aa56-596a4b18048c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.630497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4797c4-cb11-49e5-aa56-596a4b18048c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.630783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4797c4-cb11-49e5-aa56-596a4b18048c None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.631550 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4797c4-cb11-49e5-aa56-596a4b18048c None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:42.631862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4797c4-cb11-49e5-aa56-596a4b18048c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.633907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4797c4-cb11-49e5-aa56-596a4b18048c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.644975 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 428/841] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:42.651343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470296e7-0951-47d8-bea1-1eb6c5b615de None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.765705 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470296e7-0951-47d8-bea1-1eb6c5b615de None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:42.802670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692537d-60ed-4bd4-a256-429495fc0d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:42.803956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692537d-60ed-4bd4-a256-429495fc0d9b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:42.804497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692537d-60ed-4bd4-a256-429495fc0d9b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:42.805055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692537d-60ed-4bd4-a256-429495fc0d9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:42.817932 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692537d-60ed-4bd4-a256-429495fc0d9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:42.819812 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692537d-60ed-4bd4-a256-429495fc0d9b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-4692537d-60ed-4bd4-a256-429495fc0d9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:42.841277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:42.842319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:42.843383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:42.843383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:42.843383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:42.843383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:42.850168 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 414/842] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:42.852187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4797c4-cb11-49e5-aa56-596a4b18048c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:42.884662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:42.963899 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4797c4-cb11-49e5-aa56-596a4b18048c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:43.001726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab6b27-c177-47a3-a06f-43acb53f01ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:43.002430 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab6b27-c177-47a3-a06f-43acb53f01ea None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.002718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab6b27-c177-47a3-a06f-43acb53f01ea None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.002996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab6b27-c177-47a3-a06f-43acb53f01ea None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.003725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab6b27-c177-47a3-a06f-43acb53f01ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:43.004036 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab6b27-c177-47a3-a06f-43acb53f01ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:43.005410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab6b27-c177-47a3-a06f-43acb53f01ea None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:43.022475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 429/843] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:43.025205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692537d-60ed-4bd4-a256-429495fc0d9b None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:43.039321 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:43.140733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692537d-60ed-4bd4-a256-429495fc0d9b None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:43.184038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f197b4-4d90-476a-b007-0ec58a097817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:43.184871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f197b4-4d90-476a-b007-0ec58a097817 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.185160 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f197b4-4d90-476a-b007-0ec58a097817 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.185443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f197b4-4d90-476a-b007-0ec58a097817 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.193853 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f197b4-4d90-476a-b007-0ec58a097817 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:43.195423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f197b4-4d90-476a-b007-0ec58a097817 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-56f197b4-4d90-476a-b007-0ec58a097817 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:43.212272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:43.213273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:43.214258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:43.214258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:43.214258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:43.214258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:43.214258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:43.214258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:43.214258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:43.214258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:43.214258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:43.214258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:43.214258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 415/844] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 18:43:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:43.219618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab6b27-c177-47a3-a06f-43acb53f01ea None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:43.311612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:43.327588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab6b27-c177-47a3-a06f-43acb53f01ea None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:43.365733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd1e95-908e-4926-8c91-d7b02ce58584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:43.366446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1e95-908e-4926-8c91-d7b02ce58584 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.366736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1e95-908e-4926-8c91-d7b02ce58584 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.367016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1e95-908e-4926-8c91-d7b02ce58584 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.367927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd1e95-908e-4926-8c91-d7b02ce58584 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:43.368313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd1e95-908e-4926-8c91-d7b02ce58584 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:43.369890 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd1e95-908e-4926-8c91-d7b02ce58584 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:43.395581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 430/845] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Jul 25 18:43:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:43.400462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f197b4-4d90-476a-b007-0ec58a097817 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:43.478726 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:43.496352 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08e676-b7d1-47f2-ba19-d926032ba8c5 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.496696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08e676-b7d1-47f2-ba19-d926032ba8c5 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.496913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08e676-b7d1-47f2-ba19-d926032ba8c5 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.499892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 431/846] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:43.504316 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f304b3-926b-4d89-83ee-41eb79f827a3 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.504613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f304b3-926b-4d89-83ee-41eb79f827a3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.504887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f304b3-926b-4d89-83ee-41eb79f827a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.526041 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f197b4-4d90-476a-b007-0ec58a097817 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:43.528097 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d7f304b3-926b-4d89-83ee-41eb79f827a3 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:43.529419 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d7f304b3-926b-4d89-83ee-41eb79f827a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:43.569005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cea8cc-d39e-477e-998e-923564420f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:43.569747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cea8cc-d39e-477e-998e-923564420f14 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.570080 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cea8cc-d39e-477e-998e-923564420f14 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.570875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cea8cc-d39e-477e-998e-923564420f14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.634452 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 432/847] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:43.665104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f304b3-926b-4d89-83ee-41eb79f827a3 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:43.665361 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cea8cc-d39e-477e-998e-923564420f14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:43.666915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cea8cc-d39e-477e-998e-923564420f14 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:43.734928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:43.766383 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:43.788479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 416/848] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:43:43.790722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cea8cc-d39e-477e-998e-923564420f14 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:43.813333 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f304b3-926b-4d89-83ee-41eb79f827a3 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:43.873173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f0db5-9057-4db6-b35f-f4e9fe9b0e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:43.874776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f0db5-9057-4db6-b35f-f4e9fe9b0e80 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:43.875040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f0db5-9057-4db6-b35f-f4e9fe9b0e80 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:43.875512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f0db5-9057-4db6-b35f-f4e9fe9b0e80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:43.893454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f0db5-9057-4db6-b35f-f4e9fe9b0e80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-803f0db5-9057-4db6-b35f-f4e9fe9b0e80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:43.908807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:43.910743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:43.912433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:43.912433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:43.912433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:43.912433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:43.912433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:43.912433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:43.912433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:43.912433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:43.912433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:43.912433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:43.912433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:43:43.912433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:43.918640 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 433/849] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:43.921477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f0db5-9057-4db6-b35f-f4e9fe9b0e80 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:43.987188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cea8cc-d39e-477e-998e-923564420f14 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.037655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d05b0-04ad-4498-ac2c-9d687b45ae70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.040686 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d05b0-04ad-4498-ac2c-9d687b45ae70 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.041138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d05b0-04ad-4498-ac2c-9d687b45ae70 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.041912 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d05b0-04ad-4498-ac2c-9d687b45ae70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.056570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d05b0-04ad-4498-ac2c-9d687b45ae70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.058885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d05b0-04ad-4498-ac2c-9d687b45ae70 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.061675 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f0db5-9057-4db6-b35f-f4e9fe9b0e80 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-e03d05b0-04ad-4498-ac2c-9d687b45ae70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:44.083002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:44.084119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:44.085180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:44.085180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:44.085180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:44.085180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:44.088000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 417/850] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:44.095951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d05b0-04ad-4498-ac2c-9d687b45ae70 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.119075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:44.122868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f02db-056f-4ea8-b0e1-ff16395f0e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.124022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f02db-056f-4ea8-b0e1-ff16395f0e1a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.124884 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f02db-056f-4ea8-b0e1-ff16395f0e1a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.125280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f02db-056f-4ea8-b0e1-ff16395f0e1a None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.126374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f02db-056f-4ea8-b0e1-ff16395f0e1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:44.127096 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f02db-056f-4ea8-b0e1-ff16395f0e1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.129660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f02db-056f-4ea8-b0e1-ff16395f0e1a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.138992 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:44.151178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 434/851] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:44.153628 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f02db-056f-4ea8-b0e1-ff16395f0e1a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.249273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d05b0-04ad-4498-ac2c-9d687b45ae70 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.283010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f02db-056f-4ea8-b0e1-ff16395f0e1a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.304567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddffdad-1951-4394-a19d-6948888cdbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.307212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddffdad-1951-4394-a19d-6948888cdbf6 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.307212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddffdad-1951-4394-a19d-6948888cdbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.307212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddffdad-1951-4394-a19d-6948888cdbf6 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.308005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddffdad-1951-4394-a19d-6948888cdbf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:44.308605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddffdad-1951-4394-a19d-6948888cdbf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.312009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddffdad-1951-4394-a19d-6948888cdbf6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.327607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b175a-83da-43eb-ae77-341d4b1e64de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.328467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b175a-83da-43eb-ae77-341d4b1e64de None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.328786 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b175a-83da-43eb-ae77-341d4b1e64de None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.329450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b175a-83da-43eb-ae77-341d4b1e64de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.338999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 418/852] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:44.341710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddffdad-1951-4394-a19d-6948888cdbf6 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.348703 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b175a-83da-43eb-ae77-341d4b1e64de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.350609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b175a-83da-43eb-ae77-341d4b1e64de None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-3d4b175a-83da-43eb-ae77-341d4b1e64de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:44.378254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:44.387042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:44.387930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:44.387930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:44.387930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:44.387930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:44.387930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 435/853] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:44.391317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b175a-83da-43eb-ae77-341d4b1e64de None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.440752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:44.468755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:44.472917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddffdad-1951-4394-a19d-6948888cdbf6 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.518148 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce62f3-4071-4f9d-b957-56e7b0187e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.518961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce62f3-4071-4f9d-b957-56e7b0187e27 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.519679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce62f3-4071-4f9d-b957-56e7b0187e27 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.519679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce62f3-4071-4f9d-b957-56e7b0187e27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.528840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce62f3-4071-4f9d-b957-56e7b0187e27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.530450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce62f3-4071-4f9d-b957-56e7b0187e27 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.537275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b175a-83da-43eb-ae77-341d4b1e64de None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-a6ce62f3-4071-4f9d-b957-56e7b0187e27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:44.545821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:44.547134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:44.553062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:44.553062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:44.553062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:44.553062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:44.553062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:44.553062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:44.553062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:44.553062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:44.553062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 419/854] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:44.553062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9413420-f5fb-4f77-bc29-b30c262d5d56 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.553062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9413420-f5fb-4f77-bc29-b30c262d5d56 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.553755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9413420-f5fb-4f77-bc29-b30c262d5d56 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.553755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 420/855] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:44.557382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9413420-f5fb-4f77-bc29-b30c262d5d56 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.586944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7316a0-70b1-4b0c-b4c6-3b619999b152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.587621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7316a0-70b1-4b0c-b4c6-3b619999b152 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.587911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7316a0-70b1-4b0c-b4c6-3b619999b152 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.588191 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7316a0-70b1-4b0c-b4c6-3b619999b152 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.588920 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7316a0-70b1-4b0c-b4c6-3b619999b152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:44.589267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7316a0-70b1-4b0c-b4c6-3b619999b152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.590670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7316a0-70b1-4b0c-b4c6-3b619999b152 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.603990 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 436/856] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:44.605313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec9337-7ee9-4f11-9564-6e1a208c3fa9 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.605607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec9337-7ee9-4f11-9564-6e1a208c3fa9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.605887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec9337-7ee9-4f11-9564-6e1a208c3fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.629458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-7eec9337-7ee9-4f11-9564-6e1a208c3fa9 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:44.630718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7eec9337-7ee9-4f11-9564-6e1a208c3fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:44.668184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9413420-f5fb-4f77-bc29-b30c262d5d56 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.703679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 437/857] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:44.705051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496ff27-8593-48c0-83aa-5b3820dcdae4 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.705446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496ff27-8593-48c0-83aa-5b3820dcdae4 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.705732 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496ff27-8593-48c0-83aa-5b3820dcdae4 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.707057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 438/858] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:44.707768 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496ff27-8593-48c0-83aa-5b3820dcdae4 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.718361 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1204ad0-b4dc-496a-a53b-f6d77c1d64b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.719049 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1204ad0-b4dc-496a-a53b-f6d77c1d64b7 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.719709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1204ad0-b4dc-496a-a53b-f6d77c1d64b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.720203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1204ad0-b4dc-496a-a53b-f6d77c1d64b7 None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.721135 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1204ad0-b4dc-496a-a53b-f6d77c1d64b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:44.721450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1204ad0-b4dc-496a-a53b-f6d77c1d64b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.722946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1204ad0-b4dc-496a-a53b-f6d77c1d64b7 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.724880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:44.736344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 421/859] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:44.737678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea4328-36a7-4b0d-9b26-d83aad35c747 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.740995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea4328-36a7-4b0d-9b26-d83aad35c747 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.740995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea4328-36a7-4b0d-9b26-d83aad35c747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.756551 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:44.757834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e1ea4328-36a7-4b0d-9b26-d83aad35c747 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:44.760117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e1ea4328-36a7-4b0d-9b26-d83aad35c747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:44.830766 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496ff27-8593-48c0-83aa-5b3820dcdae4 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.839179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 422/860] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:44.840052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea4328-36a7-4b0d-9b26-d83aad35c747 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.870156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2226e-cf35-448f-9c15-919da79ad521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:44.871129 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2226e-cf35-448f-9c15-919da79ad521 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:44.871298 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2226e-cf35-448f-9c15-919da79ad521 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:44.871568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2226e-cf35-448f-9c15-919da79ad521 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:44.879445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2226e-cf35-448f-9c15-919da79ad521 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:44.880912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2226e-cf35-448f-9c15-919da79ad521 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-6ed2226e-cf35-448f-9c15-919da79ad521 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:44.896789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:44.897933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:44.899058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:44.899058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:44.899058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:44.899058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:44.899058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:44.899058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:44.899058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:44.899058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:44.899058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 18:43:44.899058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:44.899058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 439/861] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:43:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:44.899634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2226e-cf35-448f-9c15-919da79ad521 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:44.973159 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea4328-36a7-4b0d-9b26-d83aad35c747 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:44.999893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:45.009389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:45.011226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95cb28-93d1-4741-91df-8eb6fd6c2890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:45.012014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95cb28-93d1-4741-91df-8eb6fd6c2890 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:45.012303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95cb28-93d1-4741-91df-8eb6fd6c2890 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:45.012578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95cb28-93d1-4741-91df-8eb6fd6c2890 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:45.013239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95cb28-93d1-4741-91df-8eb6fd6c2890 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:45.013750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2226e-cf35-448f-9c15-919da79ad521 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:45.014654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95cb28-93d1-4741-91df-8eb6fd6c2890 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-0d95cb28-93d1-4741-91df-8eb6fd6c2890 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:45.028773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:45.029948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:45.031177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 423/862] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Jul 25 18:43:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:45.032029 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95cb28-93d1-4741-91df-8eb6fd6c2890 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:45.054677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379f2c2-8c58-4560-b5c4-fa158e109537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:45.056001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f2c2-8c58-4560-b5c4-fa158e109537 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:45.056358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f2c2-8c58-4560-b5c4-fa158e109537 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:45.056950 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f2c2-8c58-4560-b5c4-fa158e109537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:45.057604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379f2c2-8c58-4560-b5c4-fa158e109537 None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=c6a88af6e45549f1ab00036923f6ba04) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:45.057926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379f2c2-8c58-4560-b5c4-fa158e109537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:45.059404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379f2c2-8c58-4560-b5c4-fa158e109537 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:45.065912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 440/863] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:44 2022] GET /identity/v3/role_assignments?user.id=c6a88af6e45549f1ab00036923f6ba04&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:45.066974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379f2c2-8c58-4560-b5c4-fa158e109537 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:45.166835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95cb28-93d1-4741-91df-8eb6fd6c2890 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:45.207197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379f2c2-8c58-4560-b5c4-fa158e109537 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:45.208371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead94d4-3d08-472f-9830-9d2d7a18cb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:45.209069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead94d4-3d08-472f-9830-9d2d7a18cb18 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:45.209356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead94d4-3d08-472f-9830-9d2d7a18cb18 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:45.209642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead94d4-3d08-472f-9830-9d2d7a18cb18 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:45.210433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead94d4-3d08-472f-9830-9d2d7a18cb18 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:45.212570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead94d4-3d08-472f-9830-9d2d7a18cb18 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:45.252280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e4a00-c107-44fb-bed4-ea4f4f0f6053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:45.252975 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4a00-c107-44fb-bed4-ea4f4f0f6053 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:45.253260 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4a00-c107-44fb-bed4-ea4f4f0f6053 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:45.253536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4a00-c107-44fb-bed4-ea4f4f0f6053 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:45.254460 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e4a00-c107-44fb-bed4-ea4f4f0f6053 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:45.254778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e4a00-c107-44fb-bed4-ea4f4f0f6053 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:45.256237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e4a00-c107-44fb-bed4-ea4f4f0f6053 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:45.277638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 441/864] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Jul 25 18:43:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:45.278662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e4a00-c107-44fb-bed4-ea4f4f0f6053 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:45.303099 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:45.315546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 424/865] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:43:45 2022] POST /identity/v3/services => generated 251 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:45.323498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead94d4-3d08-472f-9830-9d2d7a18cb18 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:45.399107 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:45.422705 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e4a00-c107-44fb-bed4-ea4f4f0f6053 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:45.480168 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead94d4-3d08-472f-9830-9d2d7a18cb18 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:45.482609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fedd6-a251-45c8-854b-52d4d20abf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:45.483315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fedd6-a251-45c8-854b-52d4d20abf1e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:45.483668 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fedd6-a251-45c8-854b-52d4d20abf1e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:45.483947 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fedd6-a251-45c8-854b-52d4d20abf1e None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:45.484722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fedd6-a251-45c8-854b-52d4d20abf1e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:45.485039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fedd6-a251-45c8-854b-52d4d20abf1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:45.486504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fedd6-a251-45c8-854b-52d4d20abf1e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:45.493849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 442/866] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 18:43:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:45.499073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fedd6-a251-45c8-854b-52d4d20abf1e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:45.522247 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdc6c1-54cb-4f28-8916-0f14f4b7e9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:45.523266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdc6c1-54cb-4f28-8916-0f14f4b7e9b4 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:45.524408 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdc6c1-54cb-4f28-8916-0f14f4b7e9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:45.524868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdc6c1-54cb-4f28-8916-0f14f4b7e9b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:45.614583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdc6c1-54cb-4f28-8916-0f14f4b7e9b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:45.616222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdc6c1-54cb-4f28-8916-0f14f4b7e9b4 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:45.622467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:45.650341 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fedd6-a251-45c8-854b-52d4d20abf1e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:45.674802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:45.700302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8df71e-abc3-4490-9383-03c2db817841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:45.701002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8df71e-abc3-4490-9383-03c2db817841 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:45.701290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8df71e-abc3-4490-9383-03c2db817841 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:45.701571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8df71e-abc3-4490-9383-03c2db817841 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:45.702294 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8df71e-abc3-4490-9383-03c2db817841 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:45.702637 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8df71e-abc3-4490-9383-03c2db817841 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:45.704041 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8df71e-abc3-4490-9383-03c2db817841 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:45.711725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 443/867] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 18:43:45 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:45.739704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 425/868] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:45 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:43:45.745598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8df71e-abc3-4490-9383-03c2db817841 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:45.871840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8df71e-abc3-4490-9383-03c2db817841 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:45.922653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee27732-a172-402f-bd57-116f39ae78f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:45.923707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee27732-a172-402f-bd57-116f39ae78f9 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:45.924166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee27732-a172-402f-bd57-116f39ae78f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:45.924588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee27732-a172-402f-bd57-116f39ae78f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:45.937401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee27732-a172-402f-bd57-116f39ae78f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:45.938983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee27732-a172-402f-bd57-116f39ae78f9 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:45.942621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-cee27732-a172-402f-bd57-116f39ae78f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:45.959563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:45.960710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:45.962138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:45.962138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:45.962138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:45.962138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:45.962138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 444/869] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:45.969469 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdc6c1-54cb-4f28-8916-0f14f4b7e9b4 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:46.083628 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdc6c1-54cb-4f28-8916-0f14f4b7e9b4 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:46.122894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5bfc5-f681-4d97-ba5f-b8aed3542f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:46.124145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5bfc5-f681-4d97-ba5f-b8aed3542f72 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:46.124636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5bfc5-f681-4d97-ba5f-b8aed3542f72 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:46.125290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5bfc5-f681-4d97-ba5f-b8aed3542f72 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:46.126229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5bfc5-f681-4d97-ba5f-b8aed3542f72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:46.126749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5bfc5-f681-4d97-ba5f-b8aed3542f72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:46.128307 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5bfc5-f681-4d97-ba5f-b8aed3542f72 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:46.146438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 426/870] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:46.148298 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee27732-a172-402f-bd57-116f39ae78f9 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:46.263914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee27732-a172-402f-bd57-116f39ae78f9 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:46.307190 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11268cd-2c69-4779-b4f6-cbf12a84785a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:46.308305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11268cd-2c69-4779-b4f6-cbf12a84785a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:46.308802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11268cd-2c69-4779-b4f6-cbf12a84785a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:46.309271 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11268cd-2c69-4779-b4f6-cbf12a84785a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:46.317224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11268cd-2c69-4779-b4f6-cbf12a84785a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:46.318982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11268cd-2c69-4779-b4f6-cbf12a84785a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-c11268cd-2c69-4779-b4f6-cbf12a84785a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:46.334047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:46.335012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:46.335895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:46.335895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:46.335895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:46.335895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:46.335895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:46.335895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:46.335895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:46.335895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:46.338736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 445/871] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:46.345014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5bfc5-f681-4d97-ba5f-b8aed3542f72 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:46.358778 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:46.460858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5bfc5-f681-4d97-ba5f-b8aed3542f72 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:46.506053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5be74-ddad-4c94-8236-8ce4ee13e793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:46.507016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5be74-ddad-4c94-8236-8ce4ee13e793 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:46.507488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5be74-ddad-4c94-8236-8ce4ee13e793 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:46.508779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5be74-ddad-4c94-8236-8ce4ee13e793 None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:46.509773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5be74-ddad-4c94-8236-8ce4ee13e793 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:46.510315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5be74-ddad-4c94-8236-8ce4ee13e793 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:46.512047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5be74-ddad-4c94-8236-8ce4ee13e793 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:46.527632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 427/872] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:46.534053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11268cd-2c69-4779-b4f6-cbf12a84785a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:46.602196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:46.654961 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11268cd-2c69-4779-b4f6-cbf12a84785a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:46.690838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8d304-23e7-41ef-99ec-4723a6ec5cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:46.691512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8d304-23e7-41ef-99ec-4723a6ec5cea None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:46.691782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8d304-23e7-41ef-99ec-4723a6ec5cea None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:46.692060 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8d304-23e7-41ef-99ec-4723a6ec5cea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:46.692886 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8d304-23e7-41ef-99ec-4723a6ec5cea None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=1ee9152572224cc499229e0fa2b81ac5) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:46.693200 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8d304-23e7-41ef-99ec-4723a6ec5cea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:46.694647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8d304-23e7-41ef-99ec-4723a6ec5cea None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:46.702274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 446/873] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:46 2022] GET /identity/v3/role_assignments?user.id=1ee9152572224cc499229e0fa2b81ac5&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:47.489586 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603152b-19dc-4ccd-9b49-460146126b7f None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.490775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603152b-19dc-4ccd-9b49-460146126b7f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.491289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603152b-19dc-4ccd-9b49-460146126b7f None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.494050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 428/874] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:47.499853 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099f100-088a-48e0-afe4-e71b833d7338 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.500163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099f100-088a-48e0-afe4-e71b833d7338 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.500438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099f100-088a-48e0-afe4-e71b833d7338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.520122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-4099f100-088a-48e0-afe4-e71b833d7338 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:47.521354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-4099f100-088a-48e0-afe4-e71b833d7338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:47.542514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e039f96-1ea1-4063-9d84-b42302655e0c None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.542805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e039f96-1ea1-4063-9d84-b42302655e0c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.543076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e039f96-1ea1-4063-9d84-b42302655e0c None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.544754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 429/875] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:47.550093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e67eb7-8629-4007-8c17-abc9ec220230 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.550435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e67eb7-8629-4007-8c17-abc9ec220230 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.550723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e67eb7-8629-4007-8c17-abc9ec220230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.569817 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-f2e67eb7-8629-4007-8c17-abc9ec220230 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:47.571102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f2e67eb7-8629-4007-8c17-abc9ec220230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:47.610581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 447/876] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:47 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:47.630273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099f100-088a-48e0-afe4-e71b833d7338 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:47.651082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:47.662371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 430/877] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:47 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:43:47.690236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e67eb7-8629-4007-8c17-abc9ec220230 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:47.701035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:47.755462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099f100-088a-48e0-afe4-e71b833d7338 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:47.794729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138eb25-36fd-4ea8-84eb-3d8703003068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:47.795587 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138eb25-36fd-4ea8-84eb-3d8703003068 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.795877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138eb25-36fd-4ea8-84eb-3d8703003068 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.796157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138eb25-36fd-4ea8-84eb-3d8703003068 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.807754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138eb25-36fd-4ea8-84eb-3d8703003068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:47.815212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e67eb7-8629-4007-8c17-abc9ec220230 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-6138eb25-36fd-4ea8-84eb-3d8703003068 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:47.815906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:47.816789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:47.817620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:47.817620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:47.817620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:47.817620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:47.817620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:47.817620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:47.817620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:47.817620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:47.817620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:47.817620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:47.817620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:43:47.817620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:47.817620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 448/878] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Jul 25 18:43:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:47.823576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138eb25-36fd-4ea8-84eb-3d8703003068 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:47.864303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf3ef6-6f01-44b1-bfe7-625db78c71c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:47.865565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf3ef6-6f01-44b1-bfe7-625db78c71c0 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.866211 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf3ef6-6f01-44b1-bfe7-625db78c71c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.866750 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf3ef6-6f01-44b1-bfe7-625db78c71c0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.868248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf3ef6-6f01-44b1-bfe7-625db78c71c0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:47.869961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf3ef6-6f01-44b1-bfe7-625db78c71c0 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-9adf3ef6-6f01-44b1-bfe7-625db78c71c0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:47.889082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:47.890118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:43:47.890988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:47.894035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 431/879] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:47.901854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf3ef6-6f01-44b1-bfe7-625db78c71c0 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:47.953779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138eb25-36fd-4ea8-84eb-3d8703003068 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:47.965990 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:47.992570 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afe3a4-8b75-4d9d-81e9-0a37f46338c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:47.994152 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afe3a4-8b75-4d9d-81e9-0a37f46338c6 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:47.994152 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afe3a4-8b75-4d9d-81e9-0a37f46338c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:47.997823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afe3a4-8b75-4d9d-81e9-0a37f46338c6 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:47.997823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afe3a4-8b75-4d9d-81e9-0a37f46338c6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:47.997823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afe3a4-8b75-4d9d-81e9-0a37f46338c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:47.997823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afe3a4-8b75-4d9d-81e9-0a37f46338c6 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.002735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.013014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 449/880] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 18:43:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:48.014429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afe3a4-8b75-4d9d-81e9-0a37f46338c6 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.026745 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf3ef6-6f01-44b1-bfe7-625db78c71c0 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.069580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c0b8a-5ff1-4afe-a818-927340152979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.071681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c0b8a-5ff1-4afe-a818-927340152979 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.071978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c0b8a-5ff1-4afe-a818-927340152979 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.072267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c0b8a-5ff1-4afe-a818-927340152979 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.073009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c0b8a-5ff1-4afe-a818-927340152979 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:48.073853 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c0b8a-5ff1-4afe-a818-927340152979 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.075576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c0b8a-5ff1-4afe-a818-927340152979 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.081299 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 432/881] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:48.088053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6090700-55a2-4c9d-be75-6d28bcb23a51 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.088348 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6090700-55a2-4c9d-be75-6d28bcb23a51 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.088635 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6090700-55a2-4c9d-be75-6d28bcb23a51 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.090364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 433/882] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:48.091027 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6090700-55a2-4c9d-be75-6d28bcb23a51 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.149923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afe3a4-8b75-4d9d-81e9-0a37f46338c6 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.189407 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75d192-68b2-48d8-9683-bd8aee28e263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.190187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75d192-68b2-48d8-9683-bd8aee28e263 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.190503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75d192-68b2-48d8-9683-bd8aee28e263 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.190875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75d192-68b2-48d8-9683-bd8aee28e263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.202198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75d192-68b2-48d8-9683-bd8aee28e263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.203678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75d192-68b2-48d8-9683-bd8aee28e263 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.210686 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6090700-55a2-4c9d-be75-6d28bcb23a51 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-8a75d192-68b2-48d8-9683-bd8aee28e263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:48.222898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:48.223785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:48.224653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:48.224653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:48.224653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:48.224653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:48.226503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 450/883] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:48.228139 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e555cf-b4b2-473e-88d6-5478c4ab2424 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.228417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e555cf-b4b2-473e-88d6-5478c4ab2424 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.228696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e555cf-b4b2-473e-88d6-5478c4ab2424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.245903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.248853 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-44e555cf-b4b2-473e-88d6-5478c4ab2424 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:48.250063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-44e555cf-b4b2-473e-88d6-5478c4ab2424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:48.256433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46085681-ec53-4a41-9895-80807f806c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.260957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46085681-ec53-4a41-9895-80807f806c2f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.260957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46085681-ec53-4a41-9895-80807f806c2f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.260957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46085681-ec53-4a41-9895-80807f806c2f None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.260957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46085681-ec53-4a41-9895-80807f806c2f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:48.260957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46085681-ec53-4a41-9895-80807f806c2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.260957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46085681-ec53-4a41-9895-80807f806c2f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.266119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 434/884] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:48.266907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46085681-ec53-4a41-9895-80807f806c2f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.280288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.329458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 451/885] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:48 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:48.330650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e555cf-b4b2-473e-88d6-5478c4ab2424 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.395387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46085681-ec53-4a41-9895-80807f806c2f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.436744 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45a751-bd02-40cd-a992-d37e6810d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.437553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45a751-bd02-40cd-a992-d37e6810d06b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.437838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45a751-bd02-40cd-a992-d37e6810d06b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.440215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45a751-bd02-40cd-a992-d37e6810d06b None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.440215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45a751-bd02-40cd-a992-d37e6810d06b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:48.440215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45a751-bd02-40cd-a992-d37e6810d06b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.440562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45a751-bd02-40cd-a992-d37e6810d06b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.460131 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 435/886] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:48.460131 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45a751-bd02-40cd-a992-d37e6810d06b None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.468901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e555cf-b4b2-473e-88d6-5478c4ab2424 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.512753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26388c0b-cffe-45c6-a2b9-d87e5e076997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.513427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26388c0b-cffe-45c6-a2b9-d87e5e076997 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.513712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26388c0b-cffe-45c6-a2b9-d87e5e076997 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.514100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26388c0b-cffe-45c6-a2b9-d87e5e076997 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.515137 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26388c0b-cffe-45c6-a2b9-d87e5e076997 None admin] RBAC: Adding query filter params (interface=public, service_id=72c24cd165c846c49c5cf1854cc30acf, region_id=RegionOne) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:48.515457 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26388c0b-cffe-45c6-a2b9-d87e5e076997 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.516902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26388c0b-cffe-45c6-a2b9-d87e5e076997 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.521148 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.525261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 452/887] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/endpoints?service_id=72c24cd165c846c49c5cf1854cc30acf&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:48.526114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26388c0b-cffe-45c6-a2b9-d87e5e076997 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.547642 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.584109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45a751-bd02-40cd-a992-d37e6810d06b None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.628358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efc033-2602-4dfc-9685-b573fea80db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.629073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efc033-2602-4dfc-9685-b573fea80db2 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.629355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efc033-2602-4dfc-9685-b573fea80db2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.629642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efc033-2602-4dfc-9685-b573fea80db2 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.630398 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efc033-2602-4dfc-9685-b573fea80db2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.631906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efc033-2602-4dfc-9685-b573fea80db2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.642087 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26388c0b-cffe-45c6-a2b9-d87e5e076997 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.677512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8da302-a35a-404a-bb6d-990de5610b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.678353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8da302-a35a-404a-bb6d-990de5610b1d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.678663 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8da302-a35a-404a-bb6d-990de5610b1d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.678947 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8da302-a35a-404a-bb6d-990de5610b1d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.687473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8da302-a35a-404a-bb6d-990de5610b1d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.688940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8da302-a35a-404a-bb6d-990de5610b1d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-0e8da302-a35a-404a-bb6d-990de5610b1d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:48.704455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:48.705676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:48.706650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:48.706650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:48.706650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:48.706650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:48.706650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:48.706650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:48.706650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:48.706650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:48.706650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:48.706650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:48.706650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 453/888] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:48.707240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8da302-a35a-404a-bb6d-990de5610b1d None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.773322 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 436/889] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:43:48 2022] POST /identity/v3/services => generated 269 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:48.775525 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efc033-2602-4dfc-9685-b573fea80db2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:48.798164 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.826086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8da302-a35a-404a-bb6d-990de5610b1d None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.874419 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446039c-3f16-4b93-a547-a03b3e487d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.875377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446039c-3f16-4b93-a547-a03b3e487d59 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.875812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446039c-3f16-4b93-a547-a03b3e487d59 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.876192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446039c-3f16-4b93-a547-a03b3e487d59 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.877000 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446039c-3f16-4b93-a547-a03b3e487d59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.878692 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446039c-3f16-4b93-a547-a03b3e487d59 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.886641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 454/890] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/services => generated 1268 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:43:48.911525 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efc033-2602-4dfc-9685-b573fea80db2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:48.926285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:48.930487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06e031-6d23-4ad6-a9b2-e3f7512dd5cf None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.931091 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06e031-6d23-4ad6-a9b2-e3f7512dd5cf None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.931566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06e031-6d23-4ad6-a9b2-e3f7512dd5cf None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.933466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 455/891] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:48.939828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e084607-5284-4765-80bb-04193eecc346 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.944257 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e084607-5284-4765-80bb-04193eecc346 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.944257 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e084607-5284-4765-80bb-04193eecc346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.960577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-6e084607-5284-4765-80bb-04193eecc346 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:48.962827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6e084607-5284-4765-80bb-04193eecc346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:48.967719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fec4f5-3050-4356-a979-8bfe58780fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:48.968880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fec4f5-3050-4356-a979-8bfe58780fde None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:48.969379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fec4f5-3050-4356-a979-8bfe58780fde None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:48.970062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fec4f5-3050-4356-a979-8bfe58780fde None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:48.972679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fec4f5-3050-4356-a979-8bfe58780fde None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:48.973301 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fec4f5-3050-4356-a979-8bfe58780fde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:48.975073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fec4f5-3050-4356-a979-8bfe58780fde None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:48.998935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 437/892] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Jul 25 18:43:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:49.005885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fec4f5-3050-4356-a979-8bfe58780fde None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.080058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 456/893] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:43:49.103653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e084607-5284-4765-80bb-04193eecc346 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.150861 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fec4f5-3050-4356-a979-8bfe58780fde None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.184258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:49.192226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff45be-e3d5-485e-9ca9-aa0edd8ff6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.199257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff45be-e3d5-485e-9ca9-aa0edd8ff6c9 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.199257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff45be-e3d5-485e-9ca9-aa0edd8ff6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.199257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff45be-e3d5-485e-9ca9-aa0edd8ff6c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.224140 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e084607-5284-4765-80bb-04193eecc346 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.235679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:49.263494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82962d9c-39ca-4905-a3df-96784dd89431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.264295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82962d9c-39ca-4905-a3df-96784dd89431 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.264632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82962d9c-39ca-4905-a3df-96784dd89431 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.264920 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82962d9c-39ca-4905-a3df-96784dd89431 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.265612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82962d9c-39ca-4905-a3df-96784dd89431 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.267155 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82962d9c-39ca-4905-a3df-96784dd89431 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.277579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff45be-e3d5-485e-9ca9-aa0edd8ff6c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.279448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff45be-e3d5-485e-9ca9-aa0edd8ff6c9 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-82962d9c-39ca-4905-a3df-96784dd89431 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:49.280208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:49.281385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:49.282441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 457/894] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:49.289097 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82962d9c-39ca-4905-a3df-96784dd89431 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.403265 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82962d9c-39ca-4905-a3df-96784dd89431 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.405476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 438/895] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 18:43:49.410935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff45be-e3d5-485e-9ca9-aa0edd8ff6c9 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.436729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:49.442275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828784c-c727-49e7-a73e-bc84ea8f42f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.443177 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828784c-c727-49e7-a73e-bc84ea8f42f0 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.450756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828784c-c727-49e7-a73e-bc84ea8f42f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.450756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828784c-c727-49e7-a73e-bc84ea8f42f0 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.450756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828784c-c727-49e7-a73e-bc84ea8f42f0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:49.450756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828784c-c727-49e7-a73e-bc84ea8f42f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.450756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828784c-c727-49e7-a73e-bc84ea8f42f0 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.457073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 458/896] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:49.458274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828784c-c727-49e7-a73e-bc84ea8f42f0 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.506719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:49.550976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff45be-e3d5-485e-9ca9-aa0edd8ff6c9 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.596321 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828784c-c727-49e7-a73e-bc84ea8f42f0 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.605143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b58de1-f7e5-490a-9930-759ddbc74986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.606971 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b58de1-f7e5-490a-9930-759ddbc74986 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.607349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b58de1-f7e5-490a-9930-759ddbc74986 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.608047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b58de1-f7e5-490a-9930-759ddbc74986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.626859 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b58de1-f7e5-490a-9930-759ddbc74986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.628336 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b58de1-f7e5-490a-9930-759ddbc74986 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-74b58de1-f7e5-490a-9930-759ddbc74986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:49.651200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:49.652165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:49.653468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:49.653468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:49.653468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:49.653468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:49.653468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 439/897] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:49.654013 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3048841-6edc-42cb-8389-f8cb11a79a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.654778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3048841-6edc-42cb-8389-f8cb11a79a3d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.655070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3048841-6edc-42cb-8389-f8cb11a79a3d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.655353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3048841-6edc-42cb-8389-f8cb11a79a3d None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.656085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3048841-6edc-42cb-8389-f8cb11a79a3d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:49.656398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3048841-6edc-42cb-8389-f8cb11a79a3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.657945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b58de1-f7e5-490a-9930-759ddbc74986 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.658245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3048841-6edc-42cb-8389-f8cb11a79a3d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.666376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 459/898] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:49.795150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b58de1-f7e5-490a-9930-759ddbc74986 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:49.821272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:49.837160 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109172a-2e08-419a-b1db-5d06c82eff27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:49.837843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109172a-2e08-419a-b1db-5d06c82eff27 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:49.838133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109172a-2e08-419a-b1db-5d06c82eff27 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:49.838447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109172a-2e08-419a-b1db-5d06c82eff27 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:49.839186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109172a-2e08-419a-b1db-5d06c82eff27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:49.839504 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109172a-2e08-419a-b1db-5d06c82eff27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:49.840967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109172a-2e08-419a-b1db-5d06c82eff27 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:49.854765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 440/899] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:49.860060 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3048841-6edc-42cb-8389-f8cb11a79a3d None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:49.928936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:49.974297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3048841-6edc-42cb-8389-f8cb11a79a3d None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:50.012747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51920ab6-a725-4ab8-b422-d73833e9c7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:50.014137 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51920ab6-a725-4ab8-b422-d73833e9c7ac None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:50.014664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51920ab6-a725-4ab8-b422-d73833e9c7ac None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:50.015202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51920ab6-a725-4ab8-b422-d73833e9c7ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:50.023261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51920ab6-a725-4ab8-b422-d73833e9c7ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:50.025244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51920ab6-a725-4ab8-b422-d73833e9c7ac None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-51920ab6-a725-4ab8-b422-d73833e9c7ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:50.041855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:50.042875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:50.043783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:50.043783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:50.043783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:50.043783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:50.043783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:50.043783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:50.043783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:50.043783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:50.046413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 460/900] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:50.054906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109172a-2e08-419a-b1db-5d06c82eff27 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:50.182174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109172a-2e08-419a-b1db-5d06c82eff27 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:50.221598 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df1e18-ef1b-45ca-acec-35a78e4e6b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:50.222358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df1e18-ef1b-45ca-acec-35a78e4e6b8d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:50.222646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df1e18-ef1b-45ca-acec-35a78e4e6b8d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:50.222977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df1e18-ef1b-45ca-acec-35a78e4e6b8d None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:50.223778 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df1e18-ef1b-45ca-acec-35a78e4e6b8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:50.224094 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df1e18-ef1b-45ca-acec-35a78e4e6b8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:50.225522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df1e18-ef1b-45ca-acec-35a78e4e6b8d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:50.239718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 441/901] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:50.246004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51920ab6-a725-4ab8-b422-d73833e9c7ac None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:50.354314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51920ab6-a725-4ab8-b422-d73833e9c7ac None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:50.376582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:50.393120 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323e103-7b32-4dd0-88cd-132a7fee94a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:50.393836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323e103-7b32-4dd0-88cd-132a7fee94a1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:50.394159 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323e103-7b32-4dd0-88cd-132a7fee94a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:50.394564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323e103-7b32-4dd0-88cd-132a7fee94a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:50.395468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323e103-7b32-4dd0-88cd-132a7fee94a1 None admin] RBAC: Adding query filter params (role.id=8ccdb5d0cdab43deaae10d310e7578df, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=c6a88af6e45549f1ab00036923f6ba04) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:50.395802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323e103-7b32-4dd0-88cd-132a7fee94a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:50.397340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323e103-7b32-4dd0-88cd-132a7fee94a1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:50.404256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 461/902] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:43:50 2022] GET /identity/v3/role_assignments?user.id=c6a88af6e45549f1ab00036923f6ba04&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=8ccdb5d0cdab43deaae10d310e7578df => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:51.075956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86f9fd-c11d-4813-a77c-0f3dc22ba5f2 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.076638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86f9fd-c11d-4813-a77c-0f3dc22ba5f2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.076638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86f9fd-c11d-4813-a77c-0f3dc22ba5f2 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.078532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 442/903] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:51.079054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7783eb-4f43-481e-9166-f5f794436b50 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.079878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7783eb-4f43-481e-9166-f5f794436b50 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.080381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7783eb-4f43-481e-9166-f5f794436b50 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.082312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 462/904] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:51.083858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc72c1-7006-4fbf-b260-52c2520f7c1e None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.084370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc72c1-7006-4fbf-b260-52c2520f7c1e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.084854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc72c1-7006-4fbf-b260-52c2520f7c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.087537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42141e93-da1e-47f2-84f5-61cf3a773f64 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.087834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42141e93-da1e-47f2-84f5-61cf3a773f64 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.088118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42141e93-da1e-47f2-84f5-61cf3a773f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.106643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-92fc72c1-7006-4fbf-b260-52c2520f7c1e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:51.108234 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-92fc72c1-7006-4fbf-b260-52c2520f7c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:51.109779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-42141e93-da1e-47f2-84f5-61cf3a773f64 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:51.111036 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-42141e93-da1e-47f2-84f5-61cf3a773f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:51.150933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:51.194583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 443/905] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:51.212594 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 463/906] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:51.214051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc72c1-7006-4fbf-b260-52c2520f7c1e None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.237457 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42141e93-da1e-47f2-84f5-61cf3a773f64 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.336745 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc72c1-7006-4fbf-b260-52c2520f7c1e None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.370456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42141e93-da1e-47f2-84f5-61cf3a773f64 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.374726 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0c5ac-7a55-4e01-b15e-f3b313528e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:51.375558 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0c5ac-7a55-4e01-b15e-f3b313528e5b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.375847 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0c5ac-7a55-4e01-b15e-f3b313528e5b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.376129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0c5ac-7a55-4e01-b15e-f3b313528e5b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.376798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0c5ac-7a55-4e01-b15e-f3b313528e5b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:51.378186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0c5ac-7a55-4e01-b15e-f3b313528e5b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.389940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-2ae0c5ac-7a55-4e01-b15e-f3b313528e5b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:51.392058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:51.393035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:43:51.393954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:51.398455 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 444/907] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:51.399955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0c5ac-7a55-4e01-b15e-f3b313528e5b None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.420250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde974d-e4b6-481f-bb40-cde02a5518ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:51.421434 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde974d-e4b6-481f-bb40-cde02a5518ce None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.421931 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde974d-e4b6-481f-bb40-cde02a5518ce None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.422506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde974d-e4b6-481f-bb40-cde02a5518ce None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.424571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde974d-e4b6-481f-bb40-cde02a5518ce None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:51.426253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde974d-e4b6-481f-bb40-cde02a5518ce None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.439565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-6fde974d-e4b6-481f-bb40-cde02a5518ce None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:51.440527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:51.441485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 18:43:51.442300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:51.445253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 464/908] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:51.451733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde974d-e4b6-481f-bb40-cde02a5518ce None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.549794 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0c5ac-7a55-4e01-b15e-f3b313528e5b None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.592264 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde974d-e4b6-481f-bb40-cde02a5518ce None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.618051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df7b1a-17b7-4bf6-ae07-8878ea54eb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:51.619476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df7b1a-17b7-4bf6-ae07-8878ea54eb1e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.619986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df7b1a-17b7-4bf6-ae07-8878ea54eb1e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.620496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df7b1a-17b7-4bf6-ae07-8878ea54eb1e None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.622449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df7b1a-17b7-4bf6-ae07-8878ea54eb1e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:51.623076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df7b1a-17b7-4bf6-ae07-8878ea54eb1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:51.625985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df7b1a-17b7-4bf6-ae07-8878ea54eb1e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.635566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 445/909] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:51.641719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df7b1a-17b7-4bf6-ae07-8878ea54eb1e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.647787 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805fec4-931e-44a5-a1a2-eeed5d27666d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:51.648492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805fec4-931e-44a5-a1a2-eeed5d27666d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.648802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805fec4-931e-44a5-a1a2-eeed5d27666d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.649637 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805fec4-931e-44a5-a1a2-eeed5d27666d None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.650404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805fec4-931e-44a5-a1a2-eeed5d27666d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:51.650725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805fec4-931e-44a5-a1a2-eeed5d27666d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:51.652133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805fec4-931e-44a5-a1a2-eeed5d27666d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.665719 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 465/910] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:51.667455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805fec4-931e-44a5-a1a2-eeed5d27666d None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.722749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:51.767944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:51.773442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df7b1a-17b7-4bf6-ae07-8878ea54eb1e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.797114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805fec4-931e-44a5-a1a2-eeed5d27666d None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.814714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22c8c0-4fae-43a9-b142-1fb4ee0fe8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:51.815403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22c8c0-4fae-43a9-b142-1fb4ee0fe8be None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.815693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22c8c0-4fae-43a9-b142-1fb4ee0fe8be None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.816007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22c8c0-4fae-43a9-b142-1fb4ee0fe8be None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.816737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22c8c0-4fae-43a9-b142-1fb4ee0fe8be None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:51.817053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22c8c0-4fae-43a9-b142-1fb4ee0fe8be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:51.818452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22c8c0-4fae-43a9-b142-1fb4ee0fe8be None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.828037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 446/911] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:51.828037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949706f-ebfc-4271-a9da-2770235b8089 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.828037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949706f-ebfc-4271-a9da-2770235b8089 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.828037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949706f-ebfc-4271-a9da-2770235b8089 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.828037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 447/912] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:51.831827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949706f-ebfc-4271-a9da-2770235b8089 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:51.857478 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530e18a-2e87-42d0-b090-ae0db42fdf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:51.857999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530e18a-2e87-42d0-b090-ae0db42fdf14 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.858505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530e18a-2e87-42d0-b090-ae0db42fdf14 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.858709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530e18a-2e87-42d0-b090-ae0db42fdf14 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.859358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530e18a-2e87-42d0-b090-ae0db42fdf14 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:51.859641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530e18a-2e87-42d0-b090-ae0db42fdf14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:51.861183 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530e18a-2e87-42d0-b090-ae0db42fdf14 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:51.867545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 466/913] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:51.868912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68529b9-6aaa-47a4-8e63-8f979ba67383 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:51.869262 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68529b9-6aaa-47a4-8e63-8f979ba67383 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:51.869560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68529b9-6aaa-47a4-8e63-8f979ba67383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:51.890794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-e68529b9-6aaa-47a4-8e63-8f979ba67383 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:51.892988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e68529b9-6aaa-47a4-8e63-8f979ba67383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:51.959369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949706f-ebfc-4271-a9da-2770235b8089 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:51.983935 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 467/914] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:51.985175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68529b9-6aaa-47a4-8e63-8f979ba67383 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:52.013566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38303cb3-fc17-40f9-b5fe-ab42cd68fbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:52.014641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38303cb3-fc17-40f9-b5fe-ab42cd68fbe2 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.015113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38303cb3-fc17-40f9-b5fe-ab42cd68fbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.015578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38303cb3-fc17-40f9-b5fe-ab42cd68fbe2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.016393 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38303cb3-fc17-40f9-b5fe-ab42cd68fbe2 None admin] RBAC: Adding query filter params (interface=public, service_id=620bf482a15a4a19af21346309502c00, region_id=RegionOne) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:52.016710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38303cb3-fc17-40f9-b5fe-ab42cd68fbe2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:52.018182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38303cb3-fc17-40f9-b5fe-ab42cd68fbe2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:52.031188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 448/915] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 18:43:51 2022] GET /identity/v3/endpoints?service_id=620bf482a15a4a19af21346309502c00&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:52.031188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38303cb3-fc17-40f9-b5fe-ab42cd68fbe2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:52.042082 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:52.060638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:52.120882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68529b9-6aaa-47a4-8e63-8f979ba67383 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:52.163644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38303cb3-fc17-40f9-b5fe-ab42cd68fbe2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:52.172603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a92e1-0c58-4a2f-8c8a-7bf04a752a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:52.173584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a92e1-0c58-4a2f-8c8a-7bf04a752a6e None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.173746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a92e1-0c58-4a2f-8c8a-7bf04a752a6e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.174011 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a92e1-0c58-4a2f-8c8a-7bf04a752a6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.174834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a92e1-0c58-4a2f-8c8a-7bf04a752a6e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:52.176220 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a92e1-0c58-4a2f-8c8a-7bf04a752a6e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:52.204560 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3dd100-3dc0-4218-b029-fbb724fa7021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:52.205272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dd100-3dc0-4218-b029-fbb724fa7021 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.205566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dd100-3dc0-4218-b029-fbb724fa7021 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.205848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dd100-3dc0-4218-b029-fbb724fa7021 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.206629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3dd100-3dc0-4218-b029-fbb724fa7021 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:52.208119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3dd100-3dc0-4218-b029-fbb724fa7021 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:52.273846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 468/916] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 18:43:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:52.274890 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a92e1-0c58-4a2f-8c8a-7bf04a752a6e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:52.329539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 449/917] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:43:52 2022] POST /identity/v3/services => generated 241 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:52.420312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:52.440578 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a92e1-0c58-4a2f-8c8a-7bf04a752a6e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:52.496298 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fec1a1-b468-42f5-84ee-dd4f86c1e231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:52.497528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fec1a1-b468-42f5-84ee-dd4f86c1e231 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.498419 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fec1a1-b468-42f5-84ee-dd4f86c1e231 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.498919 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fec1a1-b468-42f5-84ee-dd4f86c1e231 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.499970 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fec1a1-b468-42f5-84ee-dd4f86c1e231 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:52.501752 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fec1a1-b468-42f5-84ee-dd4f86c1e231 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:52.509858 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 469/918] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 18:43:52 2022] GET /identity/v3/services => generated 1497 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:43:52.553091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc9c42-610d-4c1b-a843-2c2697de7b9f None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.553595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc9c42-610d-4c1b-a843-2c2697de7b9f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.553992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc9c42-610d-4c1b-a843-2c2697de7b9f None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.557218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 450/919] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:52.563184 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccca010-a705-4acc-b24a-6fba3b3d09df None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.563474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccca010-a705-4acc-b24a-6fba3b3d09df None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.563762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccca010-a705-4acc-b24a-6fba3b3d09df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.585241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-3ccca010-a705-4acc-b24a-6fba3b3d09df None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:52.586610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3ccca010-a705-4acc-b24a-6fba3b3d09df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:52.670304 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 470/920] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:52.694896 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc9c42-610d-4c1b-a843-2c2697de7b9f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:52.802546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:52.820733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc9c42-610d-4c1b-a843-2c2697de7b9f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:52.861462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53f852-6cd7-40b2-90cd-3e7387237758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:52.862741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53f852-6cd7-40b2-90cd-3e7387237758 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:52.863502 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53f852-6cd7-40b2-90cd-3e7387237758 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:52.864018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53f852-6cd7-40b2-90cd-3e7387237758 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:52.877269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53f852-6cd7-40b2-90cd-3e7387237758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-5c53f852-6cd7-40b2-90cd-3e7387237758 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:52.887239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:52.888253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:52.889079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:52.889079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:52.889079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:52.889079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:52.889079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:52.889079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:52.889079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:52.889079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:52.889079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:52.889079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:52.889079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:43:52.889079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:52.891582 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 451/921] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Jul 25 18:43:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:52.897938 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccca010-a705-4acc-b24a-6fba3b3d09df None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:52.982720 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:53.007946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccca010-a705-4acc-b24a-6fba3b3d09df None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:53.054316 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d4c15-4b32-4ae9-88d2-675dd52b7882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:53.055033 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d4c15-4b32-4ae9-88d2-675dd52b7882 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:53.055869 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d4c15-4b32-4ae9-88d2-675dd52b7882 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:53.056150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d4c15-4b32-4ae9-88d2-675dd52b7882 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:53.056913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d4c15-4b32-4ae9-88d2-675dd52b7882 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:53.057226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d4c15-4b32-4ae9-88d2-675dd52b7882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:53.058636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d4c15-4b32-4ae9-88d2-675dd52b7882 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:53.068851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 471/922] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 18:43:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:53.075111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53f852-6cd7-40b2-90cd-3e7387237758 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:53.194522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53f852-6cd7-40b2-90cd-3e7387237758 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:53.237018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a7c4b-d973-44fd-be48-73607c6a32d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:53.238206 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a7c4b-d973-44fd-be48-73607c6a32d8 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:53.238725 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a7c4b-d973-44fd-be48-73607c6a32d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:53.239203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a7c4b-d973-44fd-be48-73607c6a32d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:53.252620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a7c4b-d973-44fd-be48-73607c6a32d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:53.254447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a7c4b-d973-44fd-be48-73607c6a32d8 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:53.261915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-fb3a7c4b-d973-44fd-be48-73607c6a32d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:53.274367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:53.275655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:53.276576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:53.276576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:53.276576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:53.276576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:53.278878 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 452/923] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:53.287650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d4c15-4b32-4ae9-88d2-675dd52b7882 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:53.403863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d4c15-4b32-4ae9-88d2-675dd52b7882 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:53.441504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e4ea5-9cc4-4fa5-8912-3efbdd4785b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:53.442186 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e4ea5-9cc4-4fa5-8912-3efbdd4785b6 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:53.442508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e4ea5-9cc4-4fa5-8912-3efbdd4785b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:53.442792 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e4ea5-9cc4-4fa5-8912-3efbdd4785b6 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:53.443520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e4ea5-9cc4-4fa5-8912-3efbdd4785b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:53.443830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e4ea5-9cc4-4fa5-8912-3efbdd4785b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:53.445208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e4ea5-9cc4-4fa5-8912-3efbdd4785b6 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:53.456450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:53.460411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 472/924] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:53.467562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a7c4b-d973-44fd-be48-73607c6a32d8 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:53.593709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a7c4b-d973-44fd-be48-73607c6a32d8 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:53.633362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcce7e-b02a-4455-a888-b3d04f715bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:53.634658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcce7e-b02a-4455-a888-b3d04f715bc3 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:53.635182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcce7e-b02a-4455-a888-b3d04f715bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:53.635709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcce7e-b02a-4455-a888-b3d04f715bc3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:53.645332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcce7e-b02a-4455-a888-b3d04f715bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:53.647088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcce7e-b02a-4455-a888-b3d04f715bc3 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-59dcce7e-b02a-4455-a888-b3d04f715bc3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:53.663447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:53.664451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:53.666415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:53.666415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:53.666415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:53.666415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:53.666415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:53.666415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:53.666415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:53.666415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:53.666415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:53.666415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:53.668623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 453/925] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 18:43:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:53.674550 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e4ea5-9cc4-4fa5-8912-3efbdd4785b6 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:53.785225 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e4ea5-9cc4-4fa5-8912-3efbdd4785b6 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:53.822165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61180a61-49f0-42ed-8535-9e792ab4094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:53.822882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61180a61-49f0-42ed-8535-9e792ab4094d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:53.823168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61180a61-49f0-42ed-8535-9e792ab4094d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:53.823446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61180a61-49f0-42ed-8535-9e792ab4094d None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:53.824288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61180a61-49f0-42ed-8535-9e792ab4094d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:53.824600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61180a61-49f0-42ed-8535-9e792ab4094d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:53.826040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61180a61-49f0-42ed-8535-9e792ab4094d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:53.849442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 473/926] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Jul 25 18:43:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:53.851152 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcce7e-b02a-4455-a888-b3d04f715bc3 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:53.920834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:53.969167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcce7e-b02a-4455-a888-b3d04f715bc3 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:54.008809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef7df7-10bc-417f-a4c8-3827ed3095d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:54.010283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef7df7-10bc-417f-a4c8-3827ed3095d8 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.010816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef7df7-10bc-417f-a4c8-3827ed3095d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.011302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef7df7-10bc-417f-a4c8-3827ed3095d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.023448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef7df7-10bc-417f-a4c8-3827ed3095d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.025269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef7df7-10bc-417f-a4c8-3827ed3095d8 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-0bef7df7-10bc-417f-a4c8-3827ed3095d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:54.045288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:54.046249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:54.047807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:54.047807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:54.047807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:54.047807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:54.050472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 454/927] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:54.059559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61180a61-49f0-42ed-8535-9e792ab4094d None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:54.130147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:54.182417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61180a61-49f0-42ed-8535-9e792ab4094d None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:54.222022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde3248-03fb-4113-aa90-2a6bf6959977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:54.222845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde3248-03fb-4113-aa90-2a6bf6959977 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.223228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde3248-03fb-4113-aa90-2a6bf6959977 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.223514 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde3248-03fb-4113-aa90-2a6bf6959977 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.224242 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde3248-03fb-4113-aa90-2a6bf6959977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:54.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde3248-03fb-4113-aa90-2a6bf6959977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.225959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde3248-03fb-4113-aa90-2a6bf6959977 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:54.240228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 474/928] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:54.247430 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef7df7-10bc-417f-a4c8-3827ed3095d8 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:54.370406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef7df7-10bc-417f-a4c8-3827ed3095d8 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:54.391575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:54.410490 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6772e2-13cb-4a1f-944f-475c7c7ba1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:54.411689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6772e2-13cb-4a1f-944f-475c7c7ba1e5 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.412213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6772e2-13cb-4a1f-944f-475c7c7ba1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.412699 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6772e2-13cb-4a1f-944f-475c7c7ba1e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.421144 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6772e2-13cb-4a1f-944f-475c7c7ba1e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.423210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6772e2-13cb-4a1f-944f-475c7c7ba1e5 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-9b6772e2-13cb-4a1f-944f-475c7c7ba1e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:54.439943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:54.440884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:54.441788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:54.441788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:54.441788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:54.441788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:54.441788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:54.441788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:54.441788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:54.441788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:54.444495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 455/929] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:54.454174 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde3248-03fb-4113-aa90-2a6bf6959977 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:54.496360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1550e8-2ecd-43d6-b438-192fdd2d86ae None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.497008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1550e8-2ecd-43d6-b438-192fdd2d86ae None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.497492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1550e8-2ecd-43d6-b438-192fdd2d86ae None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.499411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 456/930] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:43:54.505216 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87a79d-225e-4f94-8fd1-764f40e30934 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.505838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87a79d-225e-4f94-8fd1-764f40e30934 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.506345 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87a79d-225e-4f94-8fd1-764f40e30934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.527059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-4e87a79d-225e-4f94-8fd1-764f40e30934 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:54.528822 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4e87a79d-225e-4f94-8fd1-764f40e30934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:54.573150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde3248-03fb-4113-aa90-2a6bf6959977 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:54.593893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:54.618356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777fca4-21af-4f20-9b68-8efec7324b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:54.618356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777fca4-21af-4f20-9b68-8efec7324b2e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.618356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777fca4-21af-4f20-9b68-8efec7324b2e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.618356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777fca4-21af-4f20-9b68-8efec7324b2e None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.619353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777fca4-21af-4f20-9b68-8efec7324b2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:54.619353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777fca4-21af-4f20-9b68-8efec7324b2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.619537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 457/931] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:54.620013 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777fca4-21af-4f20-9b68-8efec7324b2e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:54.620969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea58a92-a2d3-4e85-b227-155868ca68a8 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.621268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea58a92-a2d3-4e85-b227-155868ca68a8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.621556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea58a92-a2d3-4e85-b227-155868ca68a8 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.623891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 458/932] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:54.630218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e5f41-17a9-47cc-91ca-7b27cd972548 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.630985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e5f41-17a9-47cc-91ca-7b27cd972548 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.631266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e5f41-17a9-47cc-91ca-7b27cd972548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.634849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 475/933] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:54.638599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777fca4-21af-4f20-9b68-8efec7324b2e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:54.658935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-8b2e5f41-17a9-47cc-91ca-7b27cd972548 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:54.660433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8b2e5f41-17a9-47cc-91ca-7b27cd972548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:54.742214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 459/934] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:54.743743 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e5f41-17a9-47cc-91ca-7b27cd972548 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:54.761879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777fca4-21af-4f20-9b68-8efec7324b2e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:54.793327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:54.803382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b250ad3-be61-4fc3-88ed-d09a7a4cf7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:54.809700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b250ad3-be61-4fc3-88ed-d09a7a4cf7e9 None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.809700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b250ad3-be61-4fc3-88ed-d09a7a4cf7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.809700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b250ad3-be61-4fc3-88ed-d09a7a4cf7e9 None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.809700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b250ad3-be61-4fc3-88ed-d09a7a4cf7e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.809700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b250ad3-be61-4fc3-88ed-d09a7a4cf7e9 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:54.810717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: WARNING keystone.server.flask.common [None req-4b250ad3-be61-4fc3-88ed-d09a7a4cf7e9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 18:43:54.871911 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e5f41-17a9-47cc-91ca-7b27cd972548 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:54.872963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.sql.core [None req-4b250ad3-be61-4fc3-88ed-d09a7a4cf7e9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 18:43:54.872963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 18:43:54.872963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [parameters: {'id': '258bac6c594e4366bb059caf5950700e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 18:43:54.872963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86905) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 18:43:54.880868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-4b250ad3-be61-4fc3-88ed-d09a7a4cf7e9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 258bac6c594e4366bb059caf5950700e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 258bac6c594e4366bb059caf5950700e. Jul 25 18:43:54.880868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:54.880868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 18:43:54.880868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 18:43:54.880868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 18:43:54.880868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 18:43:54.880868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 18:43:54.880868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 18:43:54.880868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 18:43:54.880868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application conn.query(q) Jul 25 18:43:54.880868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 18:43:54.880868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 18:43:54.880868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application result.read() Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 25 18:43:54.881699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application next(self.gen) Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application yield resource Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application next(self.gen) Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise self.value Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application session.commit() Jul 25 18:43:54.882826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self.session.flush() Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application compat.raise_( Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 18:43:54.884017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 18:43:54.885004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application conn.query(q) Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application result.read() Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 18:43:54.885771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [parameters: {'id': '258bac6c594e4366bb059caf5950700e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:54.886680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:54.887928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:54.889038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:54.889038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 25 18:43:54.889038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 25 18:43:54.889038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:54.889038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:54.889038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 25 18:43:54.889038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 25 18:43:54.889038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 258bac6c594e4366bb059caf5950700e. Jul 25 18:43:54.889038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:54.889038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 476/935] 10.222.0.17 () {60 vars in 1140 bytes} [Mon Jul 25 18:43:54 2022] POST /identity/v3/projects => generated 262 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:54.889038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b250ad3-be61-4fc3-88ed-d09a7a4cf7e9 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:54.924140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d85bfe-89bf-4e09-9526-245cfc700157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:54.924140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d85bfe-89bf-4e09-9526-245cfc700157 None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:54.924140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d85bfe-89bf-4e09-9526-245cfc700157 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:54.924140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d85bfe-89bf-4e09-9526-245cfc700157 None admin] PATH_INFO: `/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/c6a88af6e45549f1ab00036923f6ba04/roles/8ccdb5d0cdab43deaae10d310e7578df` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:54.924626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d85bfe-89bf-4e09-9526-245cfc700157 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d7c3b139f284435aea4ff08a2f0952b, user_id=c6a88af6e45549f1ab00036923f6ba04, role_id=8ccdb5d0cdab43deaae10d310e7578df)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:54.924626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d85bfe-89bf-4e09-9526-245cfc700157 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:54.927156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:54.991915 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b250ad3-be61-4fc3-88ed-d09a7a4cf7e9 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.019416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 460/936] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:43:54 2022] PUT /identity/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/c6a88af6e45549f1ab00036923f6ba04/roles/8ccdb5d0cdab43deaae10d310e7578df => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:43:55.020544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d85bfe-89bf-4e09-9526-245cfc700157 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.031157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb0cc7-b806-43fe-bdba-b273f3f4bb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.031863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb0cc7-b806-43fe-bdba-b273f3f4bb30 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.032295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb0cc7-b806-43fe-bdba-b273f3f4bb30 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.032729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb0cc7-b806-43fe-bdba-b273f3f4bb30 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.033780 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb0cc7-b806-43fe-bdba-b273f3f4bb30 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.036091 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb0cc7-b806-43fe-bdba-b273f3f4bb30 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-31bb0cc7-b806-43fe-bdba-b273f3f4bb30 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:55.053004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:55.061700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:55.063378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 477/937] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 18:43:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:55.064256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6abec-aecc-4ac8-ad21-3aa1f76f9d82 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.064256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6abec-aecc-4ac8-ad21-3aa1f76f9d82 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.064256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6abec-aecc-4ac8-ad21-3aa1f76f9d82 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.064256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 478/938] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:55.064256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6abec-aecc-4ac8-ad21-3aa1f76f9d82 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.168279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:55.180914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d85bfe-89bf-4e09-9526-245cfc700157 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.205539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:55.211169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6abec-aecc-4ac8-ad21-3aa1f76f9d82 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.226199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7af9d4-dc17-40e8-8c6e-d0f3e1123e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.228091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7af9d4-dc17-40e8-8c6e-d0f3e1123e2a None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.228531 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7af9d4-dc17-40e8-8c6e-d0f3e1123e2a None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.228848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7af9d4-dc17-40e8-8c6e-d0f3e1123e2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.238170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7af9d4-dc17-40e8-8c6e-d0f3e1123e2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.240287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7af9d4-dc17-40e8-8c6e-d0f3e1123e2a None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.254100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90936992-d8a1-46de-afbe-f4270da80f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.255100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90936992-d8a1-46de-afbe-f4270da80f9e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.255559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90936992-d8a1-46de-afbe-f4270da80f9e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.255988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90936992-d8a1-46de-afbe-f4270da80f9e None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.256849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90936992-d8a1-46de-afbe-f4270da80f9e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-cb7af9d4-dc17-40e8-8c6e-d0f3e1123e2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:55.257314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:55.258689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:55.259780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:55.259780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:55.259780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:55.259780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:55.259780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:55.259780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:55.259780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:55.259780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:55.260227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90936992-d8a1-46de-afbe-f4270da80f9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.260298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 461/939] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:55.260298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6c22b-34c4-43b2-b3d9-9aa934a63125 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.260429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90936992-d8a1-46de-afbe-f4270da80f9e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.260509 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6c22b-34c4-43b2-b3d9-9aa934a63125 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.261049 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6c22b-34c4-43b2-b3d9-9aa934a63125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.266575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 479/940] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:55.267914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90936992-d8a1-46de-afbe-f4270da80f9e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.282747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-49a6c22b-34c4-43b2-b3d9-9aa934a63125 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:55.283962 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-49a6c22b-34c4-43b2-b3d9-9aa934a63125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:55.365473 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 462/941] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:55.371276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6c22b-34c4-43b2-b3d9-9aa934a63125 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.394131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90936992-d8a1-46de-afbe-f4270da80f9e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.434064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69172b2c-1807-4db4-a39c-601e58b11781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.434701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69172b2c-1807-4db4-a39c-601e58b11781 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.434991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69172b2c-1807-4db4-a39c-601e58b11781 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.435378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69172b2c-1807-4db4-a39c-601e58b11781 None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.436339 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69172b2c-1807-4db4-a39c-601e58b11781 None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:55.436861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69172b2c-1807-4db4-a39c-601e58b11781 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.439157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69172b2c-1807-4db4-a39c-601e58b11781 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.469988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 480/942] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:55.469988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69172b2c-1807-4db4-a39c-601e58b11781 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.480467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:55.512225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6c22b-34c4-43b2-b3d9-9aa934a63125 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.515762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:55.564674 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b655f97-bb6f-469d-a7ea-24770770a7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.565550 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b655f97-bb6f-469d-a7ea-24770770a7d9 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.565953 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b655f97-bb6f-469d-a7ea-24770770a7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.566664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b655f97-bb6f-469d-a7ea-24770770a7d9 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.568308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b655f97-bb6f-469d-a7ea-24770770a7d9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:55.568620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b655f97-bb6f-469d-a7ea-24770770a7d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.571807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b655f97-bb6f-469d-a7ea-24770770a7d9 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.578526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 463/943] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:55.588507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b655f97-bb6f-469d-a7ea-24770770a7d9 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.629900 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69172b2c-1807-4db4-a39c-601e58b11781 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.680911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4837204-f019-4773-8c2c-a7c289111055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.682515 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4837204-f019-4773-8c2c-a7c289111055 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.683287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4837204-f019-4773-8c2c-a7c289111055 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.683836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4837204-f019-4773-8c2c-a7c289111055 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.684755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4837204-f019-4773-8c2c-a7c289111055 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.687027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4837204-f019-4773-8c2c-a7c289111055 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-c4837204-f019-4773-8c2c-a7c289111055 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:55.704502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:55.705545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 18:43:55.706416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:55.707914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 481/944] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:55.715536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4837204-f019-4773-8c2c-a7c289111055 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.732944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b655f97-bb6f-469d-a7ea-24770770a7d9 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.785869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5137f3-3ed6-4668-a5db-8bd1bd55062c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.794413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5137f3-3ed6-4668-a5db-8bd1bd55062c None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.794413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5137f3-3ed6-4668-a5db-8bd1bd55062c None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.794413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5137f3-3ed6-4668-a5db-8bd1bd55062c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.794413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5137f3-3ed6-4668-a5db-8bd1bd55062c None admin] RBAC: Adding query filter params (interface=public, service_id=826d7e1968d44f0fbc7745930da1b52d, region_id=RegionOne) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:55.794413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5137f3-3ed6-4668-a5db-8bd1bd55062c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.794413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5137f3-3ed6-4668-a5db-8bd1bd55062c None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.799716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 464/945] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/endpoints?service_id=826d7e1968d44f0fbc7745930da1b52d&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:55.807633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5137f3-3ed6-4668-a5db-8bd1bd55062c None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.825736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:55.837480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:55.859683 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4837204-f019-4773-8c2c-a7c289111055 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.902581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e4558-4e90-428d-8854-094921ead03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:55.903731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e4558-4e90-428d-8854-094921ead03e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:55.904270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e4558-4e90-428d-8854-094921ead03e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:55.904754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e4558-4e90-428d-8854-094921ead03e None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:55.905859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e4558-4e90-428d-8854-094921ead03e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:55.906413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e4558-4e90-428d-8854-094921ead03e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:55.908032 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e4558-4e90-428d-8854-094921ead03e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:55.914544 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 482/946] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:55.920998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e4558-4e90-428d-8854-094921ead03e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:55.960325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5137f3-3ed6-4668-a5db-8bd1bd55062c None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:55.999939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06ad83-39e1-4279-86ba-3f7af808a096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:56.000725 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06ad83-39e1-4279-86ba-3f7af808a096 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.001128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06ad83-39e1-4279-86ba-3f7af808a096 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.001539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06ad83-39e1-4279-86ba-3f7af808a096 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.002362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06ad83-39e1-4279-86ba-3f7af808a096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:56.003846 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06ad83-39e1-4279-86ba-3f7af808a096 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:56.011969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 465/947] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/services => generated 1497 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:43:56.046151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e4558-4e90-428d-8854-094921ead03e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:56.089710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61ef2b-a6eb-4d8b-87ed-0fd2c2bf4621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:56.090743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61ef2b-a6eb-4d8b-87ed-0fd2c2bf4621 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.091194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61ef2b-a6eb-4d8b-87ed-0fd2c2bf4621 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.091649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61ef2b-a6eb-4d8b-87ed-0fd2c2bf4621 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.092527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61ef2b-a6eb-4d8b-87ed-0fd2c2bf4621 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:56.092983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61ef2b-a6eb-4d8b-87ed-0fd2c2bf4621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:56.094568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61ef2b-a6eb-4d8b-87ed-0fd2c2bf4621 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:56.100976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 483/948] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 18:43:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:43:56.107553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06ad83-39e1-4279-86ba-3f7af808a096 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:56.215260 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:56.251083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06ad83-39e1-4279-86ba-3f7af808a096 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:56.292445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ea12f-1e28-4414-8a7f-0d2ebd683c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:56.293106 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ea12f-1e28-4414-8a7f-0d2ebd683c07 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:56.293394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ea12f-1e28-4414-8a7f-0d2ebd683c07 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:56.293672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ea12f-1e28-4414-8a7f-0d2ebd683c07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:56.294454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ea12f-1e28-4414-8a7f-0d2ebd683c07 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:56.296165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ea12f-1e28-4414-8a7f-0d2ebd683c07 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:56.439685 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 466/949] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 18:43:56 2022] POST /identity/v3/endpoints => generated 347 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:57.187053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23454c-9365-49e8-8ddc-c6829cdc4276 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.188039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23454c-9365-49e8-8ddc-c6829cdc4276 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.188533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23454c-9365-49e8-8ddc-c6829cdc4276 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.190732 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 484/950] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:57.196371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32660457-e608-4e87-92ca-2ebb46757763 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.196672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32660457-e608-4e87-92ca-2ebb46757763 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.196957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32660457-e608-4e87-92ca-2ebb46757763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.217716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-32660457-e608-4e87-92ca-2ebb46757763 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:57.219098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-32660457-e608-4e87-92ca-2ebb46757763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:57.314128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 467/951] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:43:57.338674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23454c-9365-49e8-8ddc-c6829cdc4276 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:57.360425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:57.467572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23454c-9365-49e8-8ddc-c6829cdc4276 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:57.507843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a52a53-0e44-43a7-9481-0449b96be70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:57.508718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a52a53-0e44-43a7-9481-0449b96be70e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.509064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a52a53-0e44-43a7-9481-0449b96be70e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.509347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a52a53-0e44-43a7-9481-0449b96be70e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.521494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a52a53-0e44-43a7-9481-0449b96be70e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-e1a52a53-0e44-43a7-9481-0449b96be70e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:43:57.530832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:57.531959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:57.532904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:57.532904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:57.532904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:57.532904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:57.532904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:43:57.532904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:43:57.532904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:43:57.532904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:43:57.532904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:43:57.532904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:43:57.532904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:43:57.532904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:57.537111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 485/952] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Jul 25 18:43:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:57.539447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32660457-e608-4e87-92ca-2ebb46757763 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:57.599240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:57.650854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32660457-e608-4e87-92ca-2ebb46757763 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:57.690492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de20ec8-993f-4697-9f15-09a866488bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:57.691184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de20ec8-993f-4697-9f15-09a866488bee None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.691481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de20ec8-993f-4697-9f15-09a866488bee None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.691765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de20ec8-993f-4697-9f15-09a866488bee None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.692522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de20ec8-993f-4697-9f15-09a866488bee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:57.692836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de20ec8-993f-4697-9f15-09a866488bee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:57.694221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de20ec8-993f-4697-9f15-09a866488bee None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:57.705387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 468/953] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 18:43:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:57.711182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a52a53-0e44-43a7-9481-0449b96be70e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:57.806801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:57.849496 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a52a53-0e44-43a7-9481-0449b96be70e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:57.873541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda6aa6-c908-46d9-9806-8f8692a32503 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.873833 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda6aa6-c908-46d9-9806-8f8692a32503 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.874119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda6aa6-c908-46d9-9806-8f8692a32503 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.878443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 469/954] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:57.881291 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7685f43-3b41-4135-bc59-6c72776772df None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.881587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7685f43-3b41-4135-bc59-6c72776772df None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.881872 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7685f43-3b41-4135-bc59-6c72776772df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.897004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a088f-210c-4917-9986-4e48a95c0d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:57.897826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a088f-210c-4917-9986-4e48a95c0d3e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:57.898115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a088f-210c-4917-9986-4e48a95c0d3e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:57.898432 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a088f-210c-4917-9986-4e48a95c0d3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:57.905206 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e7685f43-3b41-4135-bc59-6c72776772df None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:57.906883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e7685f43-3b41-4135-bc59-6c72776772df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:57.912095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a088f-210c-4917-9986-4e48a95c0d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:57.913562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a088f-210c-4917-9986-4e48a95c0d3e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-b21a088f-210c-4917-9986-4e48a95c0d3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:57.936584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:57.937903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:57.939044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:57.939044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:57.939044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:57.939044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:57.939496 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 486/955] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:57.945282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a088f-210c-4917-9986-4e48a95c0d3e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:57.997873 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 470/956] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:43:58.022796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7685f43-3b41-4135-bc59-6c72776772df None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.084389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a088f-210c-4917-9986-4e48a95c0d3e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.085680 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.112772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.122053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd9582-874a-4f27-831d-3f3b12e52ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.122889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd9582-874a-4f27-831d-3f3b12e52ccc None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.123275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd9582-874a-4f27-831d-3f3b12e52ccc None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.123651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd9582-874a-4f27-831d-3f3b12e52ccc None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.124630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd9582-874a-4f27-831d-3f3b12e52ccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:58.124876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd9582-874a-4f27-831d-3f3b12e52ccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.126577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd9582-874a-4f27-831d-3f3b12e52ccc None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.152016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 487/957] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:58.152212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7685f43-3b41-4135-bc59-6c72776772df None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.156920 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd9582-874a-4f27-831d-3f3b12e52ccc None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.208083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252e25e-6cdd-4925-b213-e042d27daa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.208809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252e25e-6cdd-4925-b213-e042d27daa48 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.209143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252e25e-6cdd-4925-b213-e042d27daa48 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.209522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252e25e-6cdd-4925-b213-e042d27daa48 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.210241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252e25e-6cdd-4925-b213-e042d27daa48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.212391 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252e25e-6cdd-4925-b213-e042d27daa48 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.216268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: WARNING keystone.server.flask.common [None req-4252e25e-6cdd-4925-b213-e042d27daa48 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 18:43:58.264698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.sql.core [None req-4252e25e-6cdd-4925-b213-e042d27daa48 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 18:43:58.264698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 18:43:58.264698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [parameters: {'user_id': 'ac30f59dd1814c9b89469b25faf05ff1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 18:43:58.264698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86906) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 18:43:58.282491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd9582-874a-4f27-831d-3f3b12e52ccc None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-4252e25e-6cdd-4925-b213-e042d27daa48 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application conn.query(q) Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application result.read() Jul 25 18:43:58.283327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 18:43:58.284324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application next(self.gen) Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application yield resource Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application next(self.gen) Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise self.value Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application session.commit() Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 18:43:58.285491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self.session.flush() Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application compat.raise_( Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 18:43:58.286400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 18:43:58.287308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application conn.query(q) Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application result.read() Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 18:43:58.288136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ac30f59dd1814c9b89469b25faf05ff1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:58.289077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 25 18:43:58.292086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 25 18:43:58.295285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 25 18:43:58.295285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 25 18:43:58.295285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 18:43:58.295285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 18:43:58.295285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 18:43:58.295285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:58.295285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 471/958] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Jul 25 18:43:58 2022] POST /identity/v3/users => generated 160 bytes in 263 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:58.295285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a7191-d2c0-41a2-96a9-59beb4d164c3 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.295285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a7191-d2c0-41a2-96a9-59beb4d164c3 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.295285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a7191-d2c0-41a2-96a9-59beb4d164c3 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.296862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 472/959] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:58.296862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a7191-d2c0-41a2-96a9-59beb4d164c3 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.334766 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f4618-39e8-446d-a0f3-5f8edde97837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.335954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f4618-39e8-446d-a0f3-5f8edde97837 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.336241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f4618-39e8-446d-a0f3-5f8edde97837 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.336522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f4618-39e8-446d-a0f3-5f8edde97837 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.345554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f4618-39e8-446d-a0f3-5f8edde97837 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.347058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f4618-39e8-446d-a0f3-5f8edde97837 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-512f4618-39e8-446d-a0f3-5f8edde97837 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:58.364467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:58.365494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:58.366541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:58.366541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:58.366541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:58.366541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:58.366541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:58.366541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:58.366541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:58.366541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:58.366541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 18:43:58.366541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:58.366541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 488/960] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:43:58.367336 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2979b-7cfb-49fe-aeac-0fc5a45b8fab None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.367336 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2979b-7cfb-49fe-aeac-0fc5a45b8fab None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.368104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2979b-7cfb-49fe-aeac-0fc5a45b8fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.389168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-73b2979b-7cfb-49fe-aeac-0fc5a45b8fab None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:58.390923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-73b2979b-7cfb-49fe-aeac-0fc5a45b8fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:58.428542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a7191-d2c0-41a2-96a9-59beb4d164c3 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.440675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.442074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.471322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 489/961] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:58.472218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c36f4-5f15-401c-8926-e466d02bbbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.473104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c36f4-5f15-401c-8926-e466d02bbbed None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.473435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c36f4-5f15-401c-8926-e466d02bbbed None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.473765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c36f4-5f15-401c-8926-e466d02bbbed None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.477932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2979b-7cfb-49fe-aeac-0fc5a45b8fab None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.482585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c36f4-5f15-401c-8926-e466d02bbbed None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.484405 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c36f4-5f15-401c-8926-e466d02bbbed None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-479c36f4-5f15-401c-8926-e466d02bbbed None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:43:58.504294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:58.505294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:58.506215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:58.506215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:58.506215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:43:58.506215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:43:58.506215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:43:58.506215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:43:58.506215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:43:58.506215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:43:58.506215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 18:43:58.506215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:58.509288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 473/962] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:58.510573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c36f4-5f15-401c-8926-e466d02bbbed None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.617501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2979b-7cfb-49fe-aeac-0fc5a45b8fab None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.626874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c36f4-5f15-401c-8926-e466d02bbbed None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.661205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987f991-1587-4176-8c65-3d39fa6153b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.661893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987f991-1587-4176-8c65-3d39fa6153b5 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.662176 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987f991-1587-4176-8c65-3d39fa6153b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.662503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987f991-1587-4176-8c65-3d39fa6153b5 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.663374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987f991-1587-4176-8c65-3d39fa6153b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:58.663687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987f991-1587-4176-8c65-3d39fa6153b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.665163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987f991-1587-4176-8c65-3d39fa6153b5 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.675967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b46e9d-6877-4647-8954-a0cd8dfce7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.677338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b46e9d-6877-4647-8954-a0cd8dfce7e4 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.677873 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b46e9d-6877-4647-8954-a0cd8dfce7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.678361 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b46e9d-6877-4647-8954-a0cd8dfce7e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.679317 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b46e9d-6877-4647-8954-a0cd8dfce7e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.681480 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b46e9d-6877-4647-8954-a0cd8dfce7e4 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.692844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 490/963] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:58.693883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987f991-1587-4176-8c65-3d39fa6153b5 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-e0b46e9d-6877-4647-8954-a0cd8dfce7e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:58.697871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:58.699062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:43:58.700811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:58.704442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 474/964] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:58.705367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b46e9d-6877-4647-8954-a0cd8dfce7e4 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.743049 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.747596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:58.818801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987f991-1587-4176-8c65-3d39fa6153b5 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.819542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b46e9d-6877-4647-8954-a0cd8dfce7e4 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:58.857138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110fa6a-2130-4cc6-9c59-aec0b984fa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.857828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110fa6a-2130-4cc6-9c59-aec0b984fa6e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.858117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110fa6a-2130-4cc6-9c59-aec0b984fa6e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.858427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110fa6a-2130-4cc6-9c59-aec0b984fa6e None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.859233 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110fa6a-2130-4cc6-9c59-aec0b984fa6e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:58.859511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af617b1-0d87-446d-9df4-4c61dba5d9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:58.859630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110fa6a-2130-4cc6-9c59-aec0b984fa6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.860269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af617b1-0d87-446d-9df4-4c61dba5d9e6 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:58.860462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af617b1-0d87-446d-9df4-4c61dba5d9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:58.860853 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af617b1-0d87-446d-9df4-4c61dba5d9e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:58.860970 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110fa6a-2130-4cc6-9c59-aec0b984fa6e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.899503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 491/965] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:58.900211 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110fa6a-2130-4cc6-9c59-aec0b984fa6e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:58.938436 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af617b1-0d87-446d-9df4-4c61dba5d9e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:58.939949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af617b1-0d87-446d-9df4-4c61dba5d9e6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:58.978497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:59.018632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:59.032680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110fa6a-2130-4cc6-9c59-aec0b984fa6e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.075344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 475/966] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:43:59.075571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110ac4c-9ca0-45f0-89bc-ce0987fd7548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.076127 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110ac4c-9ca0-45f0-89bc-ce0987fd7548 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.076418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110ac4c-9ca0-45f0-89bc-ce0987fd7548 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.076708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110ac4c-9ca0-45f0-89bc-ce0987fd7548 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.076967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278b893-d0ea-4833-9d37-1c07051ada85 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.077427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110ac4c-9ca0-45f0-89bc-ce0987fd7548 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:59.077748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110ac4c-9ca0-45f0-89bc-ce0987fd7548 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.079122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278b893-d0ea-4833-9d37-1c07051ada85 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.079122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278b893-d0ea-4833-9d37-1c07051ada85 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.079427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110ac4c-9ca0-45f0-89bc-ce0987fd7548 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.080906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 476/967] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:43:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:43:59.083528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278b893-d0ea-4833-9d37-1c07051ada85 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.086717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 492/968] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 18:43:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:59.088368 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0ea90-1aa7-4cda-b105-3764c22373db None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.088682 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0ea90-1aa7-4cda-b105-3764c22373db None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.088993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0ea90-1aa7-4cda-b105-3764c22373db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.108896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-0bb0ea90-1aa7-4cda-b105-3764c22373db None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:43:59.110896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-0bb0ea90-1aa7-4cda-b105-3764c22373db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:43:59.198122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 493/969] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:43:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:43:59.199098 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0ea90-1aa7-4cda-b105-3764c22373db None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.245482 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278b893-d0ea-4833-9d37-1c07051ada85 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.280252 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:59.286139 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474a7fe-d346-4991-ba7d-b068cf6a150e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.287077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474a7fe-d346-4991-ba7d-b068cf6a150e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.287365 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474a7fe-d346-4991-ba7d-b068cf6a150e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.287697 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474a7fe-d346-4991-ba7d-b068cf6a150e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.292124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:59.300259 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474a7fe-d346-4991-ba7d-b068cf6a150e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.301926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474a7fe-d346-4991-ba7d-b068cf6a150e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.308437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0ea90-1aa7-4cda-b105-3764c22373db None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-7474a7fe-d346-4991-ba7d-b068cf6a150e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:43:59.321395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:59.322732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:59.323782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:43:59.323782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:43:59.323782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:43:59.323782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:59.323782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 477/970] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:59.324088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474a7fe-d346-4991-ba7d-b068cf6a150e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.348194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33667ecb-9e93-4588-b183-fe97c761a9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.348868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33667ecb-9e93-4588-b183-fe97c761a9f5 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.349152 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33667ecb-9e93-4588-b183-fe97c761a9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.349428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33667ecb-9e93-4588-b183-fe97c761a9f5 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.350133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33667ecb-9e93-4588-b183-fe97c761a9f5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:59.350475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33667ecb-9e93-4588-b183-fe97c761a9f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.352112 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33667ecb-9e93-4588-b183-fe97c761a9f5 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.358209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 494/971] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:59.360908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33667ecb-9e93-4588-b183-fe97c761a9f5 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.442231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474a7fe-d346-4991-ba7d-b068cf6a150e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.477043 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33667ecb-9e93-4588-b183-fe97c761a9f5 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.481448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c22b3a-4956-489c-ba39-3783d1b9d821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.482266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c22b3a-4956-489c-ba39-3783d1b9d821 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.482582 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c22b3a-4956-489c-ba39-3783d1b9d821 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.482867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c22b3a-4956-489c-ba39-3783d1b9d821 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.483537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c22b3a-4956-489c-ba39-3783d1b9d821 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.484941 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c22b3a-4956-489c-ba39-3783d1b9d821 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-39c22b3a-4956-489c-ba39-3783d1b9d821 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:59.501279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:59.502530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:43:59.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 478/972] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:59.505005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c22b3a-4956-489c-ba39-3783d1b9d821 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.521444 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a9812-3733-48c5-91d7-1a4761bfe513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.522164 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a9812-3733-48c5-91d7-1a4761bfe513 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.522488 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a9812-3733-48c5-91d7-1a4761bfe513 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.522772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a9812-3733-48c5-91d7-1a4761bfe513 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.523514 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a9812-3733-48c5-91d7-1a4761bfe513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:59.523821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a9812-3733-48c5-91d7-1a4761bfe513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.525993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a9812-3733-48c5-91d7-1a4761bfe513 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.538578 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 495/973] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:59.539402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a9812-3733-48c5-91d7-1a4761bfe513 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.558156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:59.566185 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:59.624237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c22b3a-4956-489c-ba39-3783d1b9d821 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.657704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a9812-3733-48c5-91d7-1a4761bfe513 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.661279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96274629-2ba7-4e99-b1bf-cf1b02927df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.662039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96274629-2ba7-4e99-b1bf-cf1b02927df9 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.662351 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96274629-2ba7-4e99-b1bf-cf1b02927df9 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.662643 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96274629-2ba7-4e99-b1bf-cf1b02927df9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.663393 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96274629-2ba7-4e99-b1bf-cf1b02927df9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.664867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96274629-2ba7-4e99-b1bf-cf1b02927df9 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.701583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024131a-d3d4-44d8-95de-aadea1bc3cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.702105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024131a-d3d4-44d8-95de-aadea1bc3cdc None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.702428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024131a-d3d4-44d8-95de-aadea1bc3cdc None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.702715 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024131a-d3d4-44d8-95de-aadea1bc3cdc None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.703424 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024131a-d3d4-44d8-95de-aadea1bc3cdc None admin] RBAC: Adding query filter params (name=compute) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:59.703737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024131a-d3d4-44d8-95de-aadea1bc3cdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.705222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024131a-d3d4-44d8-95de-aadea1bc3cdc None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.710288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 496/974] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:43:59.711184 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024131a-d3d4-44d8-95de-aadea1bc3cdc None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.766076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 479/975] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 18:43:59 2022] POST /identity/v3/endpoints => generated 349 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:43:59.766076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96274629-2ba7-4e99-b1bf-cf1b02927df9 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.816115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:59.849486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024131a-d3d4-44d8-95de-aadea1bc3cdc None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.891897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:43:59.898985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96274629-2ba7-4e99-b1bf-cf1b02927df9 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:43:59.901546 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eca85-bd64-467b-90c9-f2744d509db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.904696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eca85-bd64-467b-90c9-f2744d509db4 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.904696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eca85-bd64-467b-90c9-f2744d509db4 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.904696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eca85-bd64-467b-90c9-f2744d509db4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.911100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eca85-bd64-467b-90c9-f2744d509db4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.912554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eca85-bd64-467b-90c9-f2744d509db4 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-157eca85-bd64-467b-90c9-f2744d509db4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:43:59.930188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:43:59.931909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:43:59.933005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:43:59.933005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:43:59.933005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:43:59.933005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:43:59.933005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:43:59.933005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:43:59.933005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:43:59.933005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:43:59.933005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 497/976] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:43:59.938030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eca85-bd64-467b-90c9-f2744d509db4 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:43:59.951532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bdaa2-4243-44d4-b287-68df55b0bf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:43:59.953278 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bdaa2-4243-44d4-b287-68df55b0bf6e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:43:59.953501 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bdaa2-4243-44d4-b287-68df55b0bf6e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:43:59.953785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bdaa2-4243-44d4-b287-68df55b0bf6e None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:43:59.954548 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0bdaa2-4243-44d4-b287-68df55b0bf6e None admin] RBAC: Adding query filter params (type=compute) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:43:59.954926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0bdaa2-4243-44d4-b287-68df55b0bf6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:43:59.956513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0bdaa2-4243-44d4-b287-68df55b0bf6e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:43:59.964177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 480/977] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:00.062021 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eca85-bd64-467b-90c9-f2744d509db4 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:00.097402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:00.105649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460871db-10c5-4483-956c-c7b9bf11f1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:00.112276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460871db-10c5-4483-956c-c7b9bf11f1c0 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:00.112276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460871db-10c5-4483-956c-c7b9bf11f1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:00.112276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460871db-10c5-4483-956c-c7b9bf11f1c0 None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:00.112276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460871db-10c5-4483-956c-c7b9bf11f1c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:00.112276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460871db-10c5-4483-956c-c7b9bf11f1c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:00.112276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460871db-10c5-4483-956c-c7b9bf11f1c0 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:00.124160 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 498/978] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:43:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:00.129927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bdaa2-4243-44d4-b287-68df55b0bf6e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:00.291330 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bdaa2-4243-44d4-b287-68df55b0bf6e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:00.329796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e2f53-abe3-4aee-beb1-d53f4c536cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:00.330506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e2f53-abe3-4aee-beb1-d53f4c536cbc None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:00.330801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e2f53-abe3-4aee-beb1-d53f4c536cbc None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:00.331084 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e2f53-abe3-4aee-beb1-d53f4c536cbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:00.331921 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e2f53-abe3-4aee-beb1-d53f4c536cbc None admin] RBAC: Adding query filter params (role.id=8ccdb5d0cdab43deaae10d310e7578df, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=c6a88af6e45549f1ab00036923f6ba04) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:00.332862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e2f53-abe3-4aee-beb1-d53f4c536cbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:00.333674 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e2f53-abe3-4aee-beb1-d53f4c536cbc None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:00.346368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 481/979] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:00 2022] GET /identity/v3/role_assignments?user.id=c6a88af6e45549f1ab00036923f6ba04&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=8ccdb5d0cdab43deaae10d310e7578df => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:01.902681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81034981-b7dd-4b3f-b728-b32815a935d5 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:01.903280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81034981-b7dd-4b3f-b728-b32815a935d5 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:01.903280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81034981-b7dd-4b3f-b728-b32815a935d5 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:01.904797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 499/980] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:01.910528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586066e-bf31-41ee-8a13-80ed509a39fb None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:01.911087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586066e-bf31-41ee-8a13-80ed509a39fb None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:01.911087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586066e-bf31-41ee-8a13-80ed509a39fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:01.931238 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-2586066e-bf31-41ee-8a13-80ed509a39fb None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:01.933379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2586066e-bf31-41ee-8a13-80ed509a39fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:01.960507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:02.022374 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 482/981] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:02.045132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81034981-b7dd-4b3f-b728-b32815a935d5 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:02.049464 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305e296-8add-450c-94d8-c514d297ea20 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.049764 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305e296-8add-450c-94d8-c514d297ea20 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.050052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305e296-8add-450c-94d8-c514d297ea20 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.051666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 483/982] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:02.057323 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc364a-828f-4495-a43b-d5fb9517c3b5 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.057634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc364a-828f-4495-a43b-d5fb9517c3b5 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.057926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc364a-828f-4495-a43b-d5fb9517c3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.078237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-5adc364a-828f-4495-a43b-d5fb9517c3b5 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:02.079603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5adc364a-828f-4495-a43b-d5fb9517c3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:02.166412 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 484/983] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:02.167875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81034981-b7dd-4b3f-b728-b32815a935d5 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:02.184731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc364a-828f-4495-a43b-d5fb9517c3b5 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:02.205640 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff77d41-e5cd-4936-a8d9-7180668816b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:02.206505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff77d41-e5cd-4936-a8d9-7180668816b3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.206796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff77d41-e5cd-4936-a8d9-7180668816b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.207074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff77d41-e5cd-4936-a8d9-7180668816b3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.207730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff77d41-e5cd-4936-a8d9-7180668816b3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:02.209105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff77d41-e5cd-4936-a8d9-7180668816b3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-3ff77d41-e5cd-4936-a8d9-7180668816b3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:02.222453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:02.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:02.224355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 500/984] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 18:44:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:02.230758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff77d41-e5cd-4936-a8d9-7180668816b3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:02.289536 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:02.296290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:02.303953 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc364a-828f-4495-a43b-d5fb9517c3b5 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:02.345907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff77d41-e5cd-4936-a8d9-7180668816b3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:02.346280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6a280-48c1-4af9-82d5-91a020998fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:02.346906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6a280-48c1-4af9-82d5-91a020998fb4 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.347201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6a280-48c1-4af9-82d5-91a020998fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.347664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6a280-48c1-4af9-82d5-91a020998fb4 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.348432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6a280-48c1-4af9-82d5-91a020998fb4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:02.349846 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6a280-48c1-4af9-82d5-91a020998fb4 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:02.385290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38459298-784f-4d0d-b46b-e0595176e88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:02.386015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38459298-784f-4d0d-b46b-e0595176e88a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.386311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38459298-784f-4d0d-b46b-e0595176e88a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.386620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38459298-784f-4d0d-b46b-e0595176e88a None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.387321 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38459298-784f-4d0d-b46b-e0595176e88a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:02.387632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38459298-784f-4d0d-b46b-e0595176e88a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:02.389008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38459298-784f-4d0d-b46b-e0595176e88a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:02.394612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 501/985] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:02.400711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38459298-784f-4d0d-b46b-e0595176e88a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:02.474432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 485/986] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:44:02 2022] POST /identity/v3/services => generated 242 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:02.474432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6da626-b2ff-4ebe-b480-dc59331abda9 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.474432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6da626-b2ff-4ebe-b480-dc59331abda9 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.474432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6da626-b2ff-4ebe-b480-dc59331abda9 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.475065 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 486/987] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:02.484342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96292f92-f3f0-4ffe-81e4-3c16400bb769 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.484625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96292f92-f3f0-4ffe-81e4-3c16400bb769 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.485027 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96292f92-f3f0-4ffe-81e4-3c16400bb769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.509786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-96292f92-f3f0-4ffe-81e4-3c16400bb769 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:02.511098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-96292f92-f3f0-4ffe-81e4-3c16400bb769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:02.519895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38459298-784f-4d0d-b46b-e0595176e88a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:02.543472 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:02.576209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccc7cd-18b2-4141-bd54-27492d486da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:02.577103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccc7cd-18b2-4141-bd54-27492d486da6 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.577388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccc7cd-18b2-4141-bd54-27492d486da6 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.577666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccc7cd-18b2-4141-bd54-27492d486da6 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.578422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccc7cd-18b2-4141-bd54-27492d486da6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:02.578735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccc7cd-18b2-4141-bd54-27492d486da6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:02.580151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccc7cd-18b2-4141-bd54-27492d486da6 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:02.585773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 502/988] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:02.593182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccc7cd-18b2-4141-bd54-27492d486da6 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:02.602010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 487/989] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:02.625068 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96292f92-f3f0-4ffe-81e4-3c16400bb769 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:02.729412 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:02.742914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccc7cd-18b2-4141-bd54-27492d486da6 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:02.753364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96292f92-f3f0-4ffe-81e4-3c16400bb769 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:02.780895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bfdb4-904e-4dcf-90b8-448c20c8e5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:02.781582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bfdb4-904e-4dcf-90b8-448c20c8e5c6 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.781873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bfdb4-904e-4dcf-90b8-448c20c8e5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.782160 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bfdb4-904e-4dcf-90b8-448c20c8e5c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.782999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bfdb4-904e-4dcf-90b8-448c20c8e5c6 None admin] RBAC: Adding query filter params (interface=admin, service_id=826d7e1968d44f0fbc7745930da1b52d, region_id=RegionOne) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:02.783326 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bfdb4-904e-4dcf-90b8-448c20c8e5c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:02.784903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bfdb4-904e-4dcf-90b8-448c20c8e5c6 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:02.795639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 503/990] 10.222.0.17 () {56 vars in 1249 bytes} [Mon Jul 25 18:44:02 2022] GET /identity/v3/endpoints?service_id=826d7e1968d44f0fbc7745930da1b52d&interface=admin®ion_id=RegionOne => generated 193 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:02.797131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bfdb4-904e-4dcf-90b8-448c20c8e5c6 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:02.798844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954dafb-6496-4b7f-8400-5674103b959d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:02.799670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954dafb-6496-4b7f-8400-5674103b959d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.799964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954dafb-6496-4b7f-8400-5674103b959d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.800245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954dafb-6496-4b7f-8400-5674103b959d None admin] PATH_INFO: `/v3/services/image` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.801423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954dafb-6496-4b7f-8400-5674103b959d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:02.802369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954dafb-6496-4b7f-8400-5674103b959d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-1954dafb-6496-4b7f-8400-5674103b959d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:02.815140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:02.816320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:02.817528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:02.820535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 488/991] 10.222.0.17 () {56 vars in 1105 bytes} [Mon Jul 25 18:44:02 2022] GET /identity/v3/services/image => generated 86 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:02.826673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954dafb-6496-4b7f-8400-5674103b959d None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:02.858307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:02.927999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bfdb4-904e-4dcf-90b8-448c20c8e5c6 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:02.950126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954dafb-6496-4b7f-8400-5674103b959d None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:02.969049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf0c7c-af4f-4930-a83a-1f738129012e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:02.969727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0c7c-af4f-4930-a83a-1f738129012e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:02.970012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0c7c-af4f-4930-a83a-1f738129012e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:02.970287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0c7c-af4f-4930-a83a-1f738129012e None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:02.971023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf0c7c-af4f-4930-a83a-1f738129012e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:02.972404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf0c7c-af4f-4930-a83a-1f738129012e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:02.979589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 504/992] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 18:44:02 2022] GET /identity/v3/services => generated 1727 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 18:44:03.003733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c104c53-2551-49a7-8485-7b4ed612b20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:03.004435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c104c53-2551-49a7-8485-7b4ed612b20b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:03.004731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c104c53-2551-49a7-8485-7b4ed612b20b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:03.005007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c104c53-2551-49a7-8485-7b4ed612b20b None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:03.005739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c104c53-2551-49a7-8485-7b4ed612b20b None admin] RBAC: Adding query filter params (name=image) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:03.006051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c104c53-2551-49a7-8485-7b4ed612b20b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:03.007539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c104c53-2551-49a7-8485-7b4ed612b20b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:03.014905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 489/993] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:03.021248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf0c7c-af4f-4930-a83a-1f738129012e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:03.150688 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf0c7c-af4f-4930-a83a-1f738129012e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:03.161107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:03.192940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4af53-23e3-4938-bc83-795e896e3b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:03.193910 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4af53-23e3-4938-bc83-795e896e3b24 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:03.194393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4af53-23e3-4938-bc83-795e896e3b24 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:03.194957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4af53-23e3-4938-bc83-795e896e3b24 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:03.195811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4af53-23e3-4938-bc83-795e896e3b24 None admin] RBAC: Adding query filter params (type=image) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:03.197367 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4af53-23e3-4938-bc83-795e896e3b24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:03.198952 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4af53-23e3-4938-bc83-795e896e3b24 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:03.210400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 505/994] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:04.590237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc3289-781c-4cab-b589-d2714ee339e6 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:04.590816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc3289-781c-4cab-b589-d2714ee339e6 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:04.590909 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc3289-781c-4cab-b589-d2714ee339e6 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:04.592283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 490/995] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:04.597877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7481a7-c4f1-4a2b-b7f6-c8fa0f2d685a None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:04.598391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7481a7-c4f1-4a2b-b7f6-c8fa0f2d685a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:04.598486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7481a7-c4f1-4a2b-b7f6-c8fa0f2d685a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:04.619092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-bd7481a7-c4f1-4a2b-b7f6-c8fa0f2d685a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:04.620300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-bd7481a7-c4f1-4a2b-b7f6-c8fa0f2d685a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:04.695792 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 506/996] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:04 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:04.719858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc3289-781c-4cab-b589-d2714ee339e6 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:04.730992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:04.838028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc3289-781c-4cab-b589-d2714ee339e6 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:04.875221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224b2e6-0a02-4208-b20c-619992388b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:04.875945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224b2e6-0a02-4208-b20c-619992388b26 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:04.876231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224b2e6-0a02-4208-b20c-619992388b26 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:04.876572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224b2e6-0a02-4208-b20c-619992388b26 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:04.877290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224b2e6-0a02-4208-b20c-619992388b26 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:04.878709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224b2e6-0a02-4208-b20c-619992388b26 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-8224b2e6-0a02-4208-b20c-619992388b26 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:04.891905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:04.893060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:04.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 491/997] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:04.899786 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7481a7-c4f1-4a2b-b7f6-c8fa0f2d685a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.004652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:05.007929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7481a7-c4f1-4a2b-b7f6-c8fa0f2d685a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:05.043338 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77731456-ec6b-407f-80f2-078819d1eabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:05.044022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77731456-ec6b-407f-80f2-078819d1eabd None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.044374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77731456-ec6b-407f-80f2-078819d1eabd None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.044654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77731456-ec6b-407f-80f2-078819d1eabd None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.045376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77731456-ec6b-407f-80f2-078819d1eabd None admin] RBAC: Adding query filter params (name=compute) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:05.045688 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77731456-ec6b-407f-80f2-078819d1eabd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:05.047119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77731456-ec6b-407f-80f2-078819d1eabd None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:05.053253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 507/998] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 18:44:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:05.059964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224b2e6-0a02-4208-b20c-619992388b26 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.066269 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58787431-5b3a-46a6-8cfd-c64d0ad1c726 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.066590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58787431-5b3a-46a6-8cfd-c64d0ad1c726 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.066866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58787431-5b3a-46a6-8cfd-c64d0ad1c726 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.068408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 508/999] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:05.074562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4730f-a6ea-4341-b9a1-70394b034077 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.074876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4730f-a6ea-4341-b9a1-70394b034077 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.075119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4730f-a6ea-4341-b9a1-70394b034077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.094575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-bdb4730f-a6ea-4341-b9a1-70394b034077 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:05.095774 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-bdb4730f-a6ea-4341-b9a1-70394b034077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:05.142397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:05.184155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224b2e6-0a02-4208-b20c-619992388b26 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:05.192081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 509/1000] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:05.215808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4730f-a6ea-4341-b9a1-70394b034077 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.226549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89e50c-3982-49c4-ad9f-914fa20ac1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:05.227462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89e50c-3982-49c4-ad9f-914fa20ac1a4 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.228293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89e50c-3982-49c4-ad9f-914fa20ac1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.228577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89e50c-3982-49c4-ad9f-914fa20ac1a4 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.229350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89e50c-3982-49c4-ad9f-914fa20ac1a4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:05.229655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89e50c-3982-49c4-ad9f-914fa20ac1a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:05.231136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89e50c-3982-49c4-ad9f-914fa20ac1a4 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:05.242008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 492/1001] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 18:44:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:05.243130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89e50c-3982-49c4-ad9f-914fa20ac1a4 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.325299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:05.332100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4730f-a6ea-4341-b9a1-70394b034077 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:05.357311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89e50c-3982-49c4-ad9f-914fa20ac1a4 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:05.367663 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fedb21-0758-4c42-8daf-88f28ac0965e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:05.369100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fedb21-0758-4c42-8daf-88f28ac0965e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.369392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fedb21-0758-4c42-8daf-88f28ac0965e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.369672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fedb21-0758-4c42-8daf-88f28ac0965e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.370370 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fedb21-0758-4c42-8daf-88f28ac0965e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:05.371777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fedb21-0758-4c42-8daf-88f28ac0965e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-77fedb21-0758-4c42-8daf-88f28ac0965e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:05.384603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:05.385907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:05.387014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 510/1002] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 18:44:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:05.388016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a9f5-68a0-4244-8166-f32b4b8a5061 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.388016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a9f5-68a0-4244-8166-f32b4b8a5061 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.388273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a9f5-68a0-4244-8166-f32b4b8a5061 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.389565 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 511/1003] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:05.391804 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533a9f5-68a0-4244-8166-f32b4b8a5061 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.408127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4ac6e-2957-41f0-be66-196296bb5bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:05.408127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4ac6e-2957-41f0-be66-196296bb5bbb None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.408127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4ac6e-2957-41f0-be66-196296bb5bbb None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.408127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4ac6e-2957-41f0-be66-196296bb5bbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.408688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4ac6e-2957-41f0-be66-196296bb5bbb None admin] RBAC: Adding query filter params (interface=public, service_id=75bb8d07e8574b2799e396769c940f9f, region_id=RegionOne) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:05.408688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4ac6e-2957-41f0-be66-196296bb5bbb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:05.408688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4ac6e-2957-41f0-be66-196296bb5bbb None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:05.408688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:05.411066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 493/1004] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 18:44:05 2022] GET /identity/v3/endpoints?service_id=75bb8d07e8574b2799e396769c940f9f&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:05.412348 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61906deb-45c3-4062-aabc-8b5c864ed440 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.412640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61906deb-45c3-4062-aabc-8b5c864ed440 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.412924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61906deb-45c3-4062-aabc-8b5c864ed440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.432897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-61906deb-45c3-4062-aabc-8b5c864ed440 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:05.442696 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-61906deb-45c3-4062-aabc-8b5c864ed440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:05.506938 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533a9f5-68a0-4244-8166-f32b4b8a5061 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:05.507195 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 494/1005] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:05.507784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61906deb-45c3-4062-aabc-8b5c864ed440 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.547423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e8fdd-7881-431e-ac3d-353d348bf002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:05.548106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e8fdd-7881-431e-ac3d-353d348bf002 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.548392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e8fdd-7881-431e-ac3d-353d348bf002 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.548673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e8fdd-7881-431e-ac3d-353d348bf002 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.549435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e8fdd-7881-431e-ac3d-353d348bf002 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:05.549749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e8fdd-7881-431e-ac3d-353d348bf002 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:05.551160 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e8fdd-7881-431e-ac3d-353d348bf002 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:05.556759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 512/1006] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:05.557346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e8fdd-7881-431e-ac3d-353d348bf002 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.582500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:05.621767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61906deb-45c3-4062-aabc-8b5c864ed440 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:05.651486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:05.659391 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e94d1-dc5a-4491-a635-2097352b701f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:05.660052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e94d1-dc5a-4491-a635-2097352b701f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.660288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e94d1-dc5a-4491-a635-2097352b701f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.660575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e94d1-dc5a-4491-a635-2097352b701f None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.661263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e94d1-dc5a-4491-a635-2097352b701f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:05.662693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e94d1-dc5a-4491-a635-2097352b701f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:05.669601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e8fdd-7881-431e-ac3d-353d348bf002 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:05.670098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 495/1007] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 18:44:05 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:44:05.670869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e94d1-dc5a-4491-a635-2097352b701f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.709640 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04b77d-207e-4d15-804e-077ce014ea8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:05.710223 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04b77d-207e-4d15-804e-077ce014ea8e None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.710538 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04b77d-207e-4d15-804e-077ce014ea8e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.710823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04b77d-207e-4d15-804e-077ce014ea8e None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.711553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04b77d-207e-4d15-804e-077ce014ea8e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:05.713588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04b77d-207e-4d15-804e-077ce014ea8e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:05.808608 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e94d1-dc5a-4491-a635-2097352b701f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:05.847958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2dacd-85b3-4cfb-8264-a6bb21614f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:05.848640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2dacd-85b3-4cfb-8264-a6bb21614f48 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:05.848932 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2dacd-85b3-4cfb-8264-a6bb21614f48 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:05.849215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2dacd-85b3-4cfb-8264-a6bb21614f48 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:05.850044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2dacd-85b3-4cfb-8264-a6bb21614f48 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:05.850434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2dacd-85b3-4cfb-8264-a6bb21614f48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:05.851829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2dacd-85b3-4cfb-8264-a6bb21614f48 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:05.857692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 496/1008] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 18:44:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:05.863732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2dacd-85b3-4cfb-8264-a6bb21614f48 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:05.877247 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 513/1009] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:44:05 2022] POST /identity/v3/services => generated 242 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:05.935219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:05.987424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2dacd-85b3-4cfb-8264-a6bb21614f48 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:06.035845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afe142-7470-4bf2-9000-55f4adf72f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:06.035845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afe142-7470-4bf2-9000-55f4adf72f7a None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:06.036297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afe142-7470-4bf2-9000-55f4adf72f7a None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:06.036297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afe142-7470-4bf2-9000-55f4adf72f7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:06.037299 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afe142-7470-4bf2-9000-55f4adf72f7a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:06.038874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afe142-7470-4bf2-9000-55f4adf72f7a None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:06.170370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 497/1010] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 18:44:05 2022] POST /identity/v3/endpoints => generated 325 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:44:07.773839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a78508-f547-4ae4-aa79-2d4b145afc5a None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:07.774396 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a78508-f547-4ae4-aa79-2d4b145afc5a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:07.775115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a78508-f547-4ae4-aa79-2d4b145afc5a None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:07.776750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 514/1011] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:07.783929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb302ea-60b0-4278-be9b-21bef014af97 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:07.784507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb302ea-60b0-4278-be9b-21bef014af97 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:07.784507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb302ea-60b0-4278-be9b-21bef014af97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:07.804701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-9fb302ea-60b0-4278-be9b-21bef014af97 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:07.805905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9fb302ea-60b0-4278-be9b-21bef014af97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:07.892543 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 498/1012] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:07 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:07.916456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a78508-f547-4ae4-aa79-2d4b145afc5a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.011357 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:08.013462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24adc-15c9-47ee-aa7b-f75061e234f4 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.013462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24adc-15c9-47ee-aa7b-f75061e234f4 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.013462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24adc-15c9-47ee-aa7b-f75061e234f4 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.015350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 499/1013] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:08.022573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8015f68-d1fc-40f4-ba57-322c5c22b61b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.026811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8015f68-d1fc-40f4-ba57-322c5c22b61b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.026811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8015f68-d1fc-40f4-ba57-322c5c22b61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.035456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a78508-f547-4ae4-aa79-2d4b145afc5a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.043434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c8015f68-d1fc-40f4-ba57-322c5c22b61b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:08.044652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c8015f68-d1fc-40f4-ba57-322c5c22b61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:08.072768 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffd9f1-bf9e-4c4d-8479-3ef375be133c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:08.073583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffd9f1-bf9e-4c4d-8479-3ef375be133c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.073865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffd9f1-bf9e-4c4d-8479-3ef375be133c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.074145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffd9f1-bf9e-4c4d-8479-3ef375be133c None admin] PATH_INFO: `/v3/services/compute` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.074837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaffd9f1-bf9e-4c4d-8479-3ef375be133c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:08.076210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaffd9f1-bf9e-4c4d-8479-3ef375be133c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:08.078060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-aaffd9f1-bf9e-4c4d-8479-3ef375be133c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:08.089162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:08.090359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:08.091995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 515/1014] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:08.096318 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffd9f1-bf9e-4c4d-8479-3ef375be133c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.125974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 500/1015] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:08 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:08.157267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8015f68-d1fc-40f4-ba57-322c5c22b61b None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.222254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffd9f1-bf9e-4c4d-8479-3ef375be133c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.261343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f61ed3-448d-4564-b569-969f5d8769e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:08.261862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f61ed3-448d-4564-b569-969f5d8769e2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.262272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f61ed3-448d-4564-b569-969f5d8769e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.262590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f61ed3-448d-4564-b569-969f5d8769e2 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.263307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f61ed3-448d-4564-b569-969f5d8769e2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:08.263619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f61ed3-448d-4564-b569-969f5d8769e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:08.265001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f61ed3-448d-4564-b569-969f5d8769e2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:08.270258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 516/1016] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 18:44:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:08.271396 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbe941-dd26-48df-af93-2a7ab8ddbe59 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.271686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbe941-dd26-48df-af93-2a7ab8ddbe59 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.271851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8015f68-d1fc-40f4-ba57-322c5c22b61b None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.271964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbe941-dd26-48df-af93-2a7ab8ddbe59 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.273484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 517/1017] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:08.278878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbe941-dd26-48df-af93-2a7ab8ddbe59 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.292792 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:08.311535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9445ec2-cc83-49d2-8d55-bf4ea72334bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:08.312343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9445ec2-cc83-49d2-8d55-bf4ea72334bc None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.312683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9445ec2-cc83-49d2-8d55-bf4ea72334bc None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.313027 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9445ec2-cc83-49d2-8d55-bf4ea72334bc None admin] PATH_INFO: `/v3/services/image` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.313685 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9445ec2-cc83-49d2-8d55-bf4ea72334bc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:08.315304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9445ec2-cc83-49d2-8d55-bf4ea72334bc None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-a9445ec2-cc83-49d2-8d55-bf4ea72334bc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:08.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:08.329230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:08.330164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 501/1018] 10.222.0.17 () {56 vars in 1105 bytes} [Mon Jul 25 18:44:08 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:08.331199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec535a1c-486b-46a0-b0bf-132b24d20ec0 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.331199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec535a1c-486b-46a0-b0bf-132b24d20ec0 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.331199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec535a1c-486b-46a0-b0bf-132b24d20ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.350304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ec535a1c-486b-46a0-b0bf-132b24d20ec0 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:08.351603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ec535a1c-486b-46a0-b0bf-132b24d20ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:08.379388 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:08.392569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbe941-dd26-48df-af93-2a7ab8ddbe59 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.424795 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 502/1019] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:08 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:08.425727 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec535a1c-486b-46a0-b0bf-132b24d20ec0 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.430837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328190d-4a28-4b1d-8fa3-21c662158677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:08.431595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328190d-4a28-4b1d-8fa3-21c662158677 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.431881 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328190d-4a28-4b1d-8fa3-21c662158677 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.432309 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328190d-4a28-4b1d-8fa3-21c662158677 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.432990 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328190d-4a28-4b1d-8fa3-21c662158677 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:08.433368 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328190d-4a28-4b1d-8fa3-21c662158677 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:08.434699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328190d-4a28-4b1d-8fa3-21c662158677 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:08.444881 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 518/1020] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 18:44:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:08.447616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328190d-4a28-4b1d-8fa3-21c662158677 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.528381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:08.533877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec535a1c-486b-46a0-b0bf-132b24d20ec0 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.558585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328190d-4a28-4b1d-8fa3-21c662158677 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.570541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58c438-9ec9-421c-83da-3f24e36b6fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:08.571227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58c438-9ec9-421c-83da-3f24e36b6fb9 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.572530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58c438-9ec9-421c-83da-3f24e36b6fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.572818 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58c438-9ec9-421c-83da-3f24e36b6fb9 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.573547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58c438-9ec9-421c-83da-3f24e36b6fb9 None admin] RBAC: Adding query filter params (name=image) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:08.573862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58c438-9ec9-421c-83da-3f24e36b6fb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:08.575279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58c438-9ec9-421c-83da-3f24e36b6fb9 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:08.580452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 503/1021] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:08.581282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58c438-9ec9-421c-83da-3f24e36b6fb9 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.595888 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e54b-469c-4bec-afd2-25daea32223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:08.596560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e54b-469c-4bec-afd2-25daea32223f None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.596852 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e54b-469c-4bec-afd2-25daea32223f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.597135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e54b-469c-4bec-afd2-25daea32223f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.597875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39e54b-469c-4bec-afd2-25daea32223f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:08.599278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39e54b-469c-4bec-afd2-25daea32223f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:08.611967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:08.686594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58c438-9ec9-421c-83da-3f24e36b6fb9 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.705091 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 519/1022] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 18:44:08 2022] POST /identity/v3/endpoints => generated 334 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:44:08.706198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e54b-469c-4bec-afd2-25daea32223f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.733207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70ac6b-2d03-4777-a4ef-2cd208c8f11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:08.734093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70ac6b-2d03-4777-a4ef-2cd208c8f11f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.734411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70ac6b-2d03-4777-a4ef-2cd208c8f11f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.734695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70ac6b-2d03-4777-a4ef-2cd208c8f11f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.736189 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70ac6b-2d03-4777-a4ef-2cd208c8f11f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:08.737688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70ac6b-2d03-4777-a4ef-2cd208c8f11f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:08.739500 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 504/1023] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:08.746556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70ac6b-2d03-4777-a4ef-2cd208c8f11f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.842716 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e54b-469c-4bec-afd2-25daea32223f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.858587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:08.870707 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70ac6b-2d03-4777-a4ef-2cd208c8f11f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:08.885385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:08.893223 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6e381-30cd-412e-8e2d-348a1d2afd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:08.894650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6e381-30cd-412e-8e2d-348a1d2afd70 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.894837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6e381-30cd-412e-8e2d-348a1d2afd70 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.895337 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6e381-30cd-412e-8e2d-348a1d2afd70 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.896552 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6e381-30cd-412e-8e2d-348a1d2afd70 None admin] RBAC: Adding query filter params (type=image) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:08.897074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6e381-30cd-412e-8e2d-348a1d2afd70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:08.899543 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6e381-30cd-412e-8e2d-348a1d2afd70 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:08.907846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 520/1024] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:08.916280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6e381-30cd-412e-8e2d-348a1d2afd70 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:08.920342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad316c-ab18-42d5-add1-0b6ce6904afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:08.924506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad316c-ab18-42d5-add1-0b6ce6904afc None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:08.924506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad316c-ab18-42d5-add1-0b6ce6904afc None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:08.924506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad316c-ab18-42d5-add1-0b6ce6904afc None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:08.924506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad316c-ab18-42d5-add1-0b6ce6904afc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:08.924506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad316c-ab18-42d5-add1-0b6ce6904afc None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:09.027534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6e381-30cd-412e-8e2d-348a1d2afd70 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:09.063623 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced41157-8391-488e-98e8-6edf6af6885a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:09.064573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced41157-8391-488e-98e8-6edf6af6885a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:09.064871 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced41157-8391-488e-98e8-6edf6af6885a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:09.065154 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced41157-8391-488e-98e8-6edf6af6885a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:09.065958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced41157-8391-488e-98e8-6edf6af6885a None admin] RBAC: Adding query filter params (interface=public, service_id=7cfb466d43234f79b85fd002e22b779e, region_id=RegionOne) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:09.066273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced41157-8391-488e-98e8-6edf6af6885a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:09.067690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced41157-8391-488e-98e8-6edf6af6885a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:09.074760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 521/1025] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 18:44:08 2022] GET /identity/v3/endpoints?service_id=7cfb466d43234f79b85fd002e22b779e&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:09.080679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced41157-8391-488e-98e8-6edf6af6885a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:09.103624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 505/1026] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:44:08 2022] POST /identity/v3/projects => generated 311 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:09.162439 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:09.217261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced41157-8391-488e-98e8-6edf6af6885a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:09.261059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75bcde-1044-4d7d-9e55-ac91b5d40a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:09.262391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75bcde-1044-4d7d-9e55-ac91b5d40a06 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:09.262391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75bcde-1044-4d7d-9e55-ac91b5d40a06 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:09.262391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75bcde-1044-4d7d-9e55-ac91b5d40a06 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:09.263250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75bcde-1044-4d7d-9e55-ac91b5d40a06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:09.264744 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75bcde-1044-4d7d-9e55-ac91b5d40a06 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:09.276152 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 522/1027] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 18:44:09 2022] GET /identity/v3/services => generated 1957 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 18:44:10.908681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d63919-2d7f-46dd-863f-78bb7e98a622 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:10.909231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d63919-2d7f-46dd-863f-78bb7e98a622 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:10.909231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d63919-2d7f-46dd-863f-78bb7e98a622 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:10.911009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 506/1028] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:10.916887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ecbd9-46de-4a9d-8739-9a000420f357 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:10.917376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ecbd9-46de-4a9d-8739-9a000420f357 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:10.917571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ecbd9-46de-4a9d-8739-9a000420f357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:10.937864 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-b90ecbd9-46de-4a9d-8739-9a000420f357 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:10.939156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b90ecbd9-46de-4a9d-8739-9a000420f357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:11.014821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 523/1029] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:11.038997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d63919-2d7f-46dd-863f-78bb7e98a622 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:11.164076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d63919-2d7f-46dd-863f-78bb7e98a622 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:11.202427 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eeb71a-8f38-444e-b4e1-126b67beaa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:11.203198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eeb71a-8f38-444e-b4e1-126b67beaa09 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.203772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eeb71a-8f38-444e-b4e1-126b67beaa09 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.203772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eeb71a-8f38-444e-b4e1-126b67beaa09 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.216038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eeb71a-8f38-444e-b4e1-126b67beaa09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-64eeb71a-8f38-444e-b4e1-126b67beaa09 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:11.224585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:11.225589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:11.227042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:11.227042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:11.227042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:11.227042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:11.227042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:11.227042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:11.227042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:11.227042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:11.227042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:11.227042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:11.227042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:11.227042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:11.230869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 507/1030] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Jul 25 18:44:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:11.232558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ecbd9-46de-4a9d-8739-9a000420f357 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:11.280703 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2843d-6d15-4524-bfef-33841f8ead3f None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.280996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2843d-6d15-4524-bfef-33841f8ead3f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.281454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2843d-6d15-4524-bfef-33841f8ead3f None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.283053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 508/1031] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:11.288620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f633f06-a22e-4f73-90a0-b35ca038e73d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.288918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f633f06-a22e-4f73-90a0-b35ca038e73d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.289210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f633f06-a22e-4f73-90a0-b35ca038e73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.296577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:11.310739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-4f633f06-a22e-4f73-90a0-b35ca038e73d None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:11.312249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4f633f06-a22e-4f73-90a0-b35ca038e73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:11.316231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:11.369718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ecbd9-46de-4a9d-8739-9a000420f357 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:11.395292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 509/1032] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:11.400345 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d99f63-3017-4833-a3be-b24676e2578b None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.400646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d99f63-3017-4833-a3be-b24676e2578b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.401032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d99f63-3017-4833-a3be-b24676e2578b None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.402533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 510/1033] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:11.407891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f274e-48c1-4cf8-8f5c-9a3d9d7df087 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.408172 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f274e-48c1-4cf8-8f5c-9a3d9d7df087 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.408452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f274e-48c1-4cf8-8f5c-9a3d9d7df087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.415759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e2b16-c4c4-4276-91b5-46f7b513a692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:11.415759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e2b16-c4c4-4276-91b5-46f7b513a692 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.415759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e2b16-c4c4-4276-91b5-46f7b513a692 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.415759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e2b16-c4c4-4276-91b5-46f7b513a692 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.417608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e2b16-c4c4-4276-91b5-46f7b513a692 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:11.417608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e2b16-c4c4-4276-91b5-46f7b513a692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:11.417608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e2b16-c4c4-4276-91b5-46f7b513a692 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:11.430369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 524/1034] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 18:44:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:11.430369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e2b16-c4c4-4276-91b5-46f7b513a692 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:11.432677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-d99f274e-48c1-4cf8-8f5c-9a3d9d7df087 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:11.433943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d99f274e-48c1-4cf8-8f5c-9a3d9d7df087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:11.520671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 511/1035] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:11.522775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f274e-48c1-4cf8-8f5c-9a3d9d7df087 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:11.570436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e2b16-c4c4-4276-91b5-46f7b513a692 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:11.594651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:11.611974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:11.619771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ddcff-7858-45df-9a2c-68de62633526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:11.621172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ddcff-7858-45df-9a2c-68de62633526 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.621172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ddcff-7858-45df-9a2c-68de62633526 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.621172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ddcff-7858-45df-9a2c-68de62633526 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.622472 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ddcff-7858-45df-9a2c-68de62633526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:11.624118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ddcff-7858-45df-9a2c-68de62633526 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:11.625876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 525/1036] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:11.628292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ddcff-7858-45df-9a2c-68de62633526 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:11.667437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f274e-48c1-4cf8-8f5c-9a3d9d7df087 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:11.721694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4475e5e-5e4a-4a58-bb4f-a853fd2baea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:11.723228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4475e5e-5e4a-4a58-bb4f-a853fd2baea7 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.723228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4475e5e-5e4a-4a58-bb4f-a853fd2baea7 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.723504 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4475e5e-5e4a-4a58-bb4f-a853fd2baea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.737140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4475e5e-5e4a-4a58-bb4f-a853fd2baea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:11.738684 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4475e5e-5e4a-4a58-bb4f-a853fd2baea7 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-e4475e5e-5e4a-4a58-bb4f-a853fd2baea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:11.762165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:11.763303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:11.764440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:11.764440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:11.764440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:11.764440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:11.764440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 512/1037] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:11.764440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4475e5e-5e4a-4a58-bb4f-a853fd2baea7 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:11.769702 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ddcff-7858-45df-9a2c-68de62633526 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:11.821382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7e8c1-4a73-4549-8efc-068562ac86da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:11.822483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7e8c1-4a73-4549-8efc-068562ac86da None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.822780 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7e8c1-4a73-4549-8efc-068562ac86da None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.823179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7e8c1-4a73-4549-8efc-068562ac86da None admin] PATH_INFO: `/v3/services/image` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.823741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7e8c1-4a73-4549-8efc-068562ac86da None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:11.825281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7e8c1-4a73-4549-8efc-068562ac86da None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-40c7e8c1-4a73-4549-8efc-068562ac86da None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:11.839525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:11.841327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:11.842726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 526/1038] 10.222.0.17 () {56 vars in 1105 bytes} [Mon Jul 25 18:44:11 2022] GET /identity/v3/services/image => generated 86 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:11.843687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7e8c1-4a73-4549-8efc-068562ac86da None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:11.898879 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4475e5e-5e4a-4a58-bb4f-a853fd2baea7 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:11.918918 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:11.922126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:11.950409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49385415-5666-4c16-8400-01d96f11d0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:11.951111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49385415-5666-4c16-8400-01d96f11d0bc None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:11.951411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49385415-5666-4c16-8400-01d96f11d0bc None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:11.951687 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49385415-5666-4c16-8400-01d96f11d0bc None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:11.952534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49385415-5666-4c16-8400-01d96f11d0bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:11.953960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49385415-5666-4c16-8400-01d96f11d0bc None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.000166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7e8c1-4a73-4549-8efc-068562ac86da None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.041521 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec09667-5904-4454-a700-b00466187a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.042629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec09667-5904-4454-a700-b00466187a12 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.043114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec09667-5904-4454-a700-b00466187a12 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.043586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec09667-5904-4454-a700-b00466187a12 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.044785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec09667-5904-4454-a700-b00466187a12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:12.045313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec09667-5904-4454-a700-b00466187a12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.047220 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec09667-5904-4454-a700-b00466187a12 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.060460 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 527/1039] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:12.061337 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec09667-5904-4454-a700-b00466187a12 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.070606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 513/1040] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:44:11 2022] POST /identity/v3/users => generated 285 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:12.075569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49385415-5666-4c16-8400-01d96f11d0bc None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.216095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec09667-5904-4454-a700-b00466187a12 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.232161 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:12.260883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49385415-5666-4c16-8400-01d96f11d0bc None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.270858 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac712323-9a92-4a10-8a36-60c9486039ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.271693 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac712323-9a92-4a10-8a36-60c9486039ca None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.272031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac712323-9a92-4a10-8a36-60c9486039ca None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.272333 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac712323-9a92-4a10-8a36-60c9486039ca None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.273023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac712323-9a92-4a10-8a36-60c9486039ca None admin] RBAC: Adding query filter params (name=image) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:12.273361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac712323-9a92-4a10-8a36-60c9486039ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.274812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac712323-9a92-4a10-8a36-60c9486039ca None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.283271 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 528/1041] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:12.289793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac712323-9a92-4a10-8a36-60c9486039ca None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.316708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb4c1b-a069-497e-8d7d-1d0583bfac3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.320358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb4c1b-a069-497e-8d7d-1d0583bfac3f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.320358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb4c1b-a069-497e-8d7d-1d0583bfac3f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.320358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb4c1b-a069-497e-8d7d-1d0583bfac3f None admin] PATH_INFO: `/v3/users/nova` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.340254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb4c1b-a069-497e-8d7d-1d0583bfac3f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.343036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb4c1b-a069-497e-8d7d-1d0583bfac3f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-dfbb4c1b-a069-497e-8d7d-1d0583bfac3f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:44:12.359479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:12.360624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:12.361769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:12.361769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:12.361769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:44:12.361769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:44:12.361769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:44:12.361769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:44:12.361769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:44:12.361769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:44:12.361769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:44:12.361769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:12.361769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 514/1042] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:12.367517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb4c1b-a069-497e-8d7d-1d0583bfac3f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.415997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:12.438947 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac712323-9a92-4a10-8a36-60c9486039ca None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.491706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d58451-0f7c-440f-9e29-6e6fbbc5454a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.492332 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d58451-0f7c-440f-9e29-6e6fbbc5454a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.492632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d58451-0f7c-440f-9e29-6e6fbbc5454a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.492913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d58451-0f7c-440f-9e29-6e6fbbc5454a None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.494204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d58451-0f7c-440f-9e29-6e6fbbc5454a None admin] RBAC: Adding query filter params (type=image) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:12.494545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d58451-0f7c-440f-9e29-6e6fbbc5454a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.495976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d58451-0f7c-440f-9e29-6e6fbbc5454a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.508092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 529/1043] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:12.515142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d58451-0f7c-440f-9e29-6e6fbbc5454a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.535133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb4c1b-a069-497e-8d7d-1d0583bfac3f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.560553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:12.582942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f0d04-35d1-4a96-9bbc-f723c378aec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.583639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f0d04-35d1-4a96-9bbc-f723c378aec8 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.583943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f0d04-35d1-4a96-9bbc-f723c378aec8 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.584316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f0d04-35d1-4a96-9bbc-f723c378aec8 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.585689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f0d04-35d1-4a96-9bbc-f723c378aec8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:12.586015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f0d04-35d1-4a96-9bbc-f723c378aec8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.587769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f0d04-35d1-4a96-9bbc-f723c378aec8 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.614904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 515/1044] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:12.617021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f0d04-35d1-4a96-9bbc-f723c378aec8 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:12.661333 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d58451-0f7c-440f-9e29-6e6fbbc5454a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.717204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783c206-8cf3-4be2-a899-7e2e4b5f0152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.718198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c206-8cf3-4be2-a899-7e2e4b5f0152 None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.718718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c206-8cf3-4be2-a899-7e2e4b5f0152 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.719231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c206-8cf3-4be2-a899-7e2e4b5f0152 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.720490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6783c206-8cf3-4be2-a899-7e2e4b5f0152 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.722830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6783c206-8cf3-4be2-a899-7e2e4b5f0152 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.751302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:12.764368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f0d04-35d1-4a96-9bbc-f723c378aec8 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:12.800563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63575f97-9ddf-486d-b08f-f8c1dfb02a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:12.801224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63575f97-9ddf-486d-b08f-f8c1dfb02a67 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:12.801509 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63575f97-9ddf-486d-b08f-f8c1dfb02a67 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:12.801924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63575f97-9ddf-486d-b08f-f8c1dfb02a67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:12.873778 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63575f97-9ddf-486d-b08f-f8c1dfb02a67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:12.875279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63575f97-9ddf-486d-b08f-f8c1dfb02a67 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:12.907568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 530/1045] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 18:44:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:44:12.985617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:13.009065 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 516/1046] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:12 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:13.011071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783c206-8cf3-4be2-a899-7e2e4b5f0152 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:13.116836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:13.137448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783c206-8cf3-4be2-a899-7e2e4b5f0152 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:13.176957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d22d1-23b3-494e-ace0-d5ca9315ca4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:13.177834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d22d1-23b3-494e-ace0-d5ca9315ca4c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.182149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d22d1-23b3-494e-ace0-d5ca9315ca4c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.182149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d22d1-23b3-494e-ace0-d5ca9315ca4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.189762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d22d1-23b3-494e-ace0-d5ca9315ca4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:13.191384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d22d1-23b3-494e-ace0-d5ca9315ca4c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-ff7d22d1-23b3-494e-ace0-d5ca9315ca4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:13.210461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:13.211408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:13.212785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:13.212785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:13.212785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:13.212785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:13.214711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 531/1047] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:13.220727 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63575f97-9ddf-486d-b08f-f8c1dfb02a67 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:13.346891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63575f97-9ddf-486d-b08f-f8c1dfb02a67 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:13.433444 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f8b7b-0a77-41de-8ce0-fa2f4d3c223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:13.434228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8b7b-0a77-41de-8ce0-fa2f4d3c223f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.434613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8b7b-0a77-41de-8ce0-fa2f4d3c223f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.434901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8b7b-0a77-41de-8ce0-fa2f4d3c223f None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.435908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f8b7b-0a77-41de-8ce0-fa2f4d3c223f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:13.436038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f8b7b-0a77-41de-8ce0-fa2f4d3c223f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:13.437350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f8b7b-0a77-41de-8ce0-fa2f4d3c223f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:13.452873 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 517/1048] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:13.460014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d22d1-23b3-494e-ace0-d5ca9315ca4c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:13.576896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d22d1-23b3-494e-ace0-d5ca9315ca4c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:13.616250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb510895-a0e8-45f1-ba05-a7e70db96bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:13.616965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb510895-a0e8-45f1-ba05-a7e70db96bed None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.617249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb510895-a0e8-45f1-ba05-a7e70db96bed None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.617525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb510895-a0e8-45f1-ba05-a7e70db96bed None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.625823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb510895-a0e8-45f1-ba05-a7e70db96bed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:13.627463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb510895-a0e8-45f1-ba05-a7e70db96bed None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:13.629390 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-bb510895-a0e8-45f1-ba05-a7e70db96bed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:44:13.643582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:13.647213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:13.648113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:44:13.648113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:44:13.648113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:44:13.648113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:44:13.648113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:44:13.648113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:44:13.648113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:13.648113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:13.648113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 532/1049] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:13.651747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f8b7b-0a77-41de-8ce0-fa2f4d3c223f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:13.707790 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:13.767190 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f8b7b-0a77-41de-8ce0-fa2f4d3c223f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:13.804703 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1042a-5b81-4cd8-ae51-7457b371f00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:13.805383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1042a-5b81-4cd8-ae51-7457b371f00b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.805671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1042a-5b81-4cd8-ae51-7457b371f00b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.805951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1042a-5b81-4cd8-ae51-7457b371f00b None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.807336 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1042a-5b81-4cd8-ae51-7457b371f00b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:13.807656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1042a-5b81-4cd8-ae51-7457b371f00b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:13.809100 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1042a-5b81-4cd8-ae51-7457b371f00b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:13.833459 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 518/1050] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:44:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:13.834884 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb510895-a0e8-45f1-ba05-a7e70db96bed None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:13.952315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb510895-a0e8-45f1-ba05-a7e70db96bed None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:13.989799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838adbac-e403-4646-87ed-ef2b862a55a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:13.990703 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838adbac-e403-4646-87ed-ef2b862a55a2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:13.990996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838adbac-e403-4646-87ed-ef2b862a55a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:13.991272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838adbac-e403-4646-87ed-ef2b862a55a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:13.992128 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838adbac-e403-4646-87ed-ef2b862a55a2 None admin] RBAC: Adding query filter params (role.id=8ccdb5d0cdab43deaae10d310e7578df, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=e628cb75e0c74f27a390945c8a04335c) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:13.992438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838adbac-e403-4646-87ed-ef2b862a55a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:13.995832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838adbac-e403-4646-87ed-ef2b862a55a2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:14.010367 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 533/1051] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:13 2022] GET /identity/v3/role_assignments?user.id=e628cb75e0c74f27a390945c8a04335c&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=8ccdb5d0cdab43deaae10d310e7578df => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:14.380483 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef47d64-ef20-4644-960d-6df6e0a158cb None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:14.381009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef47d64-ef20-4644-960d-6df6e0a158cb None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:14.381009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef47d64-ef20-4644-960d-6df6e0a158cb None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:14.382530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 519/1052] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:14.388699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647a6f2-756d-48fd-b692-8021284aa82b None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:14.389219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647a6f2-756d-48fd-b692-8021284aa82b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:14.389378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647a6f2-756d-48fd-b692-8021284aa82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:14.409237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-1647a6f2-756d-48fd-b692-8021284aa82b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:14.410540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1647a6f2-756d-48fd-b692-8021284aa82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:14.480619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:14.497238 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 534/1053] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:14.518129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef47d64-ef20-4644-960d-6df6e0a158cb None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:14.630772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef47d64-ef20-4644-960d-6df6e0a158cb None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:14.641370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:14.669637 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb3a5c-6704-457e-83eb-08acc0b560a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:14.670474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb3a5c-6704-457e-83eb-08acc0b560a3 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:14.670764 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb3a5c-6704-457e-83eb-08acc0b560a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:14.671043 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb3a5c-6704-457e-83eb-08acc0b560a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:14.683185 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb3a5c-6704-457e-83eb-08acc0b560a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-6bfb3a5c-6704-457e-83eb-08acc0b560a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:14.691970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:14.693009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:14.694103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:14.694103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:14.694103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:14.694103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:14.694103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:14.694103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:14.694103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:14.694103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:14.694103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:14.694103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:14.694103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:14.694103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:14.694103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 520/1054] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:44:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:14.699775 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647a6f2-756d-48fd-b692-8021284aa82b None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:14.818661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647a6f2-756d-48fd-b692-8021284aa82b None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:14.862253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e47ca-4c49-4b0c-b097-51e1603f087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:14.863093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e47ca-4c49-4b0c-b097-51e1603f087e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:14.863388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e47ca-4c49-4b0c-b097-51e1603f087e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:14.863676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e47ca-4c49-4b0c-b097-51e1603f087e None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:14.864431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e47ca-4c49-4b0c-b097-51e1603f087e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:14.864743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e47ca-4c49-4b0c-b097-51e1603f087e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:14.866353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e47ca-4c49-4b0c-b097-51e1603f087e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:14.879983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 535/1055] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:44:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:14.889386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb3a5c-6704-457e-83eb-08acc0b560a3 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:15.007079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb3a5c-6704-457e-83eb-08acc0b560a3 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:15.043301 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74bdcd-4c69-44d5-b0be-a1cd75c16389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:15.044481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74bdcd-4c69-44d5-b0be-a1cd75c16389 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.044916 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74bdcd-4c69-44d5-b0be-a1cd75c16389 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.045371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74bdcd-4c69-44d5-b0be-a1cd75c16389 None admin] PATH_INFO: `/v3/users/8ecfd254b29047a19f9b302c6e1c4f3f` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.059828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e342f8b-d8db-4e5d-a43b-f3ec583a6bca None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.060418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e342f8b-d8db-4e5d-a43b-f3ec583a6bca None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.060936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e342f8b-d8db-4e5d-a43b-f3ec583a6bca None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.062735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74bdcd-4c69-44d5-b0be-a1cd75c16389 None admin] RBAC: Authorizing `identity:get_user(user_id=8ecfd254b29047a19f9b302c6e1c4f3f)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:15.064489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74bdcd-4c69-44d5-b0be-a1cd75c16389 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:15.067018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 536/1057] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:15.067155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 521/1057] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:14 2022] GET /identity/v3/users/8ecfd254b29047a19f9b302c6e1c4f3f => generated 285 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:15.068471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52d2f-4812-416c-be80-09a416eb25fb None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.068767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52d2f-4812-416c-be80-09a416eb25fb None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.069048 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52d2f-4812-416c-be80-09a416eb25fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.072503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e342f8b-d8db-4e5d-a43b-f3ec583a6bca None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:15.090812 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e2c52d2f-4812-416c-be80-09a416eb25fb None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:15.092607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e2c52d2f-4812-416c-be80-09a416eb25fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:15.120147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:15.163835 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:15.198444 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 522/1058] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:15.223652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c52d2f-4812-416c-be80-09a416eb25fb None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:15.232019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e342f8b-d8db-4e5d-a43b-f3ec583a6bca None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:15.271339 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa049423-6bab-4fe6-b0c7-d3b1b32efc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:15.272007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa049423-6bab-4fe6-b0c7-d3b1b32efc99 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.272311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa049423-6bab-4fe6-b0c7-d3b1b32efc99 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.272580 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa049423-6bab-4fe6-b0c7-d3b1b32efc99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.342655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c52d2f-4812-416c-be80-09a416eb25fb None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:15.353783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa049423-6bab-4fe6-b0c7-d3b1b32efc99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:15.355283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa049423-6bab-4fe6-b0c7-d3b1b32efc99 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:15.428710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfad66-6201-4355-9cb9-5c38322e642e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:15.429936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfad66-6201-4355-9cb9-5c38322e642e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.430442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfad66-6201-4355-9cb9-5c38322e642e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.430880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfad66-6201-4355-9cb9-5c38322e642e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.453998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:15.461020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:15.515915 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 537/1059] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:15.516117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfad66-6201-4355-9cb9-5c38322e642e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:15.516117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfad66-6201-4355-9cb9-5c38322e642e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:15.518055 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa049423-6bab-4fe6-b0c7-d3b1b32efc99 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:15.619973 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 523/1060] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:15.625822 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfad66-6201-4355-9cb9-5c38322e642e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:15.631243 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa049423-6bab-4fe6-b0c7-d3b1b32efc99 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:15.668504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c31df7-052d-4c42-9e4d-87ad133ca18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:15.669299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c31df7-052d-4c42-9e4d-87ad133ca18f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.669581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c31df7-052d-4c42-9e4d-87ad133ca18f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.669859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c31df7-052d-4c42-9e4d-87ad133ca18f None admin] PATH_INFO: `/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.671109 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c31df7-052d-4c42-9e4d-87ad133ca18f None admin] RBAC: Authorizing `identity:get_project(project_id=55a1b53bb39a4e6397e50ee9eff9802f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:15.672563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c31df7-052d-4c42-9e4d-87ad133ca18f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:15.679307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 538/1061] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:15 2022] GET /identity/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:15.680819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c31df7-052d-4c42-9e4d-87ad133ca18f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:15.700022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:15.716254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:15.752989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfad66-6201-4355-9cb9-5c38322e642e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:15.792435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818cc5e-a561-45fb-9561-819a5ae6cc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:15.793204 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818cc5e-a561-45fb-9561-819a5ae6cc11 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.796337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818cc5e-a561-45fb-9561-819a5ae6cc11 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.796337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818cc5e-a561-45fb-9561-819a5ae6cc11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.802166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c31df7-052d-4c42-9e4d-87ad133ca18f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:15.807470 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818cc5e-a561-45fb-9561-819a5ae6cc11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:15.808934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818cc5e-a561-45fb-9561-819a5ae6cc11 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-d818cc5e-a561-45fb-9561-819a5ae6cc11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:15.828977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:15.829942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:15.831497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:15.831497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:15.831497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:15.831497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:15.831497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 524/1062] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:15.836028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818cc5e-a561-45fb-9561-819a5ae6cc11 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:15.841284 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3663c-e87c-460d-9ea1-652f1e1df6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:15.842211 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3663c-e87c-460d-9ea1-652f1e1df6af None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:15.842540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3663c-e87c-460d-9ea1-652f1e1df6af None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:15.842815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3663c-e87c-460d-9ea1-652f1e1df6af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:15.843650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3663c-e87c-460d-9ea1-652f1e1df6af None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=55a1b53bb39a4e6397e50ee9eff9802f, user.id=8ecfd254b29047a19f9b302c6e1c4f3f) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:15.843965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3663c-e87c-460d-9ea1-652f1e1df6af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:15.845404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3663c-e87c-460d-9ea1-652f1e1df6af None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:15.854387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 539/1063] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:15 2022] GET /identity/v3/role_assignments?user.id=8ecfd254b29047a19f9b302c6e1c4f3f&scope.project.id=55a1b53bb39a4e6397e50ee9eff9802f&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:15.986667 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818cc5e-a561-45fb-9561-819a5ae6cc11 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:15.997413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:16.027292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84a4f6-1f6e-4987-aa08-659ff0c53567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:16.027967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84a4f6-1f6e-4987-aa08-659ff0c53567 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.028254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84a4f6-1f6e-4987-aa08-659ff0c53567 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.028531 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84a4f6-1f6e-4987-aa08-659ff0c53567 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.029261 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84a4f6-1f6e-4987-aa08-659ff0c53567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:16.029573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84a4f6-1f6e-4987-aa08-659ff0c53567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:16.031004 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84a4f6-1f6e-4987-aa08-659ff0c53567 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:16.045619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 525/1064] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:16.111423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afcd62-7b4c-499a-a57b-644e1b284e9e None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.111704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afcd62-7b4c-499a-a57b-644e1b284e9e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.111989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afcd62-7b4c-499a-a57b-644e1b284e9e None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.113699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 540/1065] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:16.119882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7d68b-26ff-445e-a02b-c8b8abdd7b37 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.120193 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7d68b-26ff-445e-a02b-c8b8abdd7b37 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.120462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7d68b-26ff-445e-a02b-c8b8abdd7b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.141041 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-2dc7d68b-26ff-445e-a02b-c8b8abdd7b37 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:16.142296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2dc7d68b-26ff-445e-a02b-c8b8abdd7b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:16.233645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 526/1066] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:16.258877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afcd62-7b4c-499a-a57b-644e1b284e9e None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:16.381466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:16.398568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afcd62-7b4c-499a-a57b-644e1b284e9e None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:16.437480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab04635-e4bd-472c-9b70-062df0e554ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:16.438294 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab04635-e4bd-472c-9b70-062df0e554ef None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.438610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab04635-e4bd-472c-9b70-062df0e554ef None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.438890 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab04635-e4bd-472c-9b70-062df0e554ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.449292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab04635-e4bd-472c-9b70-062df0e554ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-fab04635-e4bd-472c-9b70-062df0e554ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:16.457598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:16.459080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:16.459922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:16.459922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:16.459922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:16.459922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:16.459922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:16.459922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:16.459922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:16.459922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:16.459922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:16.459922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:16.459922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:16.459922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:16.459922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 541/1067] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Jul 25 18:44:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:16.465106 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7d68b-26ff-445e-a02b-c8b8abdd7b37 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:16.568334 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:16.572013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7d68b-26ff-445e-a02b-c8b8abdd7b37 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:16.609553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba1798-a2b4-4d13-a719-0aab08ffa0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:16.609553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba1798-a2b4-4d13-a719-0aab08ffa0db None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.609553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba1798-a2b4-4d13-a719-0aab08ffa0db None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.609553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba1798-a2b4-4d13-a719-0aab08ffa0db None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.612478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba1798-a2b4-4d13-a719-0aab08ffa0db None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:16.612478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba1798-a2b4-4d13-a719-0aab08ffa0db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:16.613712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba1798-a2b4-4d13-a719-0aab08ffa0db None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:16.629311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 527/1068] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 18:44:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:16.630801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab04635-e4bd-472c-9b70-062df0e554ef None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:16.736381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab04635-e4bd-472c-9b70-062df0e554ef None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:16.782381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7d46-3dd7-41f4-9389-f09a7afd0e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:16.783194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7d46-3dd7-41f4-9389-f09a7afd0e70 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.783478 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7d46-3dd7-41f4-9389-f09a7afd0e70 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.783758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7d46-3dd7-41f4-9389-f09a7afd0e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.796109 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c7d46-3dd7-41f4-9389-f09a7afd0e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:16.797593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c7d46-3dd7-41f4-9389-f09a7afd0e70 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:16.804489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-ae0c7d46-3dd7-41f4-9389-f09a7afd0e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:16.816487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:16.817475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:16.818956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:16.818956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:16.818956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:16.818956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:16.818956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 542/1069] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:16.824012 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba1798-a2b4-4d13-a719-0aab08ffa0db None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:16.944418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba1798-a2b4-4d13-a719-0aab08ffa0db None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:16.988275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ba053-0060-4594-b884-239668804ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:16.988956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ba053-0060-4594-b884-239668804ef2 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:16.989275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ba053-0060-4594-b884-239668804ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:16.989557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ba053-0060-4594-b884-239668804ef2 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:16.990387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ba053-0060-4594-b884-239668804ef2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:16.990703 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ba053-0060-4594-b884-239668804ef2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:16.992107 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ba053-0060-4594-b884-239668804ef2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:17.005664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 528/1070] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:17.011932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7d46-3dd7-41f4-9389-f09a7afd0e70 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:17.121780 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7d46-3dd7-41f4-9389-f09a7afd0e70 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:17.158084 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451ee9b-a9b0-4e3d-b620-1b49df51e5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:17.158934 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451ee9b-a9b0-4e3d-b620-1b49df51e5a1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:17.159442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451ee9b-a9b0-4e3d-b620-1b49df51e5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:17.159578 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451ee9b-a9b0-4e3d-b620-1b49df51e5a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:17.167877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451ee9b-a9b0-4e3d-b620-1b49df51e5a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:17.169360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451ee9b-a9b0-4e3d-b620-1b49df51e5a1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-1451ee9b-a9b0-4e3d-b620-1b49df51e5a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:44:17.183761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:17.184724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:17.185815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:17.185815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:17.185815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:44:17.185815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:44:17.185815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:44:17.185815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:44:17.185815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:44:17.185815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:44:17.185815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:44:17.185815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:17.185815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 543/1071] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 18:44:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:17.190830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ba053-0060-4594-b884-239668804ef2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:17.214255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:17.295453 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ba053-0060-4594-b884-239668804ef2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:17.332012 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7df1-3055-4210-bc92-ed4a117ec926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:17.332696 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7df1-3055-4210-bc92-ed4a117ec926 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:17.332985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7df1-3055-4210-bc92-ed4a117ec926 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:17.333265 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7df1-3055-4210-bc92-ed4a117ec926 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:17.334104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7df1-3055-4210-bc92-ed4a117ec926 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:17.334455 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7df1-3055-4210-bc92-ed4a117ec926 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:17.335898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7df1-3055-4210-bc92-ed4a117ec926 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:17.355998 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 529/1072] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 18:44:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:17.361735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451ee9b-a9b0-4e3d-b620-1b49df51e5a1 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:17.426485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:17.470559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451ee9b-a9b0-4e3d-b620-1b49df51e5a1 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:17.507287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1368c0-9deb-43b8-a5d6-3a984fa608b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:17.508085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1368c0-9deb-43b8-a5d6-3a984fa608b2 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:17.508442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1368c0-9deb-43b8-a5d6-3a984fa608b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:17.508645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1368c0-9deb-43b8-a5d6-3a984fa608b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:17.520317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1368c0-9deb-43b8-a5d6-3a984fa608b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:17.521792 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1368c0-9deb-43b8-a5d6-3a984fa608b2 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-8d1368c0-9deb-43b8-a5d6-3a984fa608b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:17.539974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:17.540956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:17.541991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:17.541991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:17.541991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:17.541991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:17.541991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 544/1073] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:17.547668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7df1-3055-4210-bc92-ed4a117ec926 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:17.630037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:17.652970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7df1-3055-4210-bc92-ed4a117ec926 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:17.688558 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bebfa-6388-4216-b513-997bfe675906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:17.689237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bebfa-6388-4216-b513-997bfe675906 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:17.689518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bebfa-6388-4216-b513-997bfe675906 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:17.689802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bebfa-6388-4216-b513-997bfe675906 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:17.690594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bebfa-6388-4216-b513-997bfe675906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:17.690948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bebfa-6388-4216-b513-997bfe675906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:17.692420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bebfa-6388-4216-b513-997bfe675906 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:17.707057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 530/1074] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:17.712804 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1368c0-9deb-43b8-a5d6-3a984fa608b2 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:17.819885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1368c0-9deb-43b8-a5d6-3a984fa608b2 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:17.831550 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:17.855685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26c8a8-c7b9-4680-a0d2-559fc22bf029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:17.856588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26c8a8-c7b9-4680-a0d2-559fc22bf029 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:17.856877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26c8a8-c7b9-4680-a0d2-559fc22bf029 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:17.857160 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26c8a8-c7b9-4680-a0d2-559fc22bf029 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:17.864396 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26c8a8-c7b9-4680-a0d2-559fc22bf029 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:17.865924 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26c8a8-c7b9-4680-a0d2-559fc22bf029 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-da26c8a8-c7b9-4680-a0d2-559fc22bf029 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:44:17.881696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:17.882677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:17.883644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:44:17.883644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:44:17.883644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:44:17.883644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:44:17.883644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:44:17.883644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:44:17.883644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:17.883644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:17.886630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 545/1075] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:17.889457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bebfa-6388-4216-b513-997bfe675906 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:17.956465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3041d-8b3c-4c01-ac97-148ee13f2096 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:17.956685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3041d-8b3c-4c01-ac97-148ee13f2096 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:17.956994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3041d-8b3c-4c01-ac97-148ee13f2096 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:17.958434 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 546/1076] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:17.963996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210a2e2-e65a-4569-a06d-fbc5dc64d36f None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:17.964289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210a2e2-e65a-4569-a06d-fbc5dc64d36f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:17.964577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210a2e2-e65a-4569-a06d-fbc5dc64d36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:17.984405 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-b210a2e2-e65a-4569-a06d-fbc5dc64d36f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:17.985650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b210a2e2-e65a-4569-a06d-fbc5dc64d36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:18.005795 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bebfa-6388-4216-b513-997bfe675906 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.045141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed6762-8724-429d-a20b-41f4ca5dac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.045828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed6762-8724-429d-a20b-41f4ca5dac54 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.046603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed6762-8724-429d-a20b-41f4ca5dac54 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.046882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed6762-8724-429d-a20b-41f4ca5dac54 None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.047680 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed6762-8724-429d-a20b-41f4ca5dac54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:18.047996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed6762-8724-429d-a20b-41f4ca5dac54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.049442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed6762-8724-429d-a20b-41f4ca5dac54 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.058111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:18.061463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 547/1077] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:18.066353 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 531/1078] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:44:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:18.069693 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210a2e2-e65a-4569-a06d-fbc5dc64d36f None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.085013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed6762-8724-429d-a20b-41f4ca5dac54 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.177482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:18.180575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210a2e2-e65a-4569-a06d-fbc5dc64d36f None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.194913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed6762-8724-429d-a20b-41f4ca5dac54 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.217092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb6407-fabf-48fe-a835-a192568291ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.217935 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb6407-fabf-48fe-a835-a192568291ce None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.218223 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb6407-fabf-48fe-a835-a192568291ce None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.219617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb6407-fabf-48fe-a835-a192568291ce None admin] PATH_INFO: `/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/e628cb75e0c74f27a390945c8a04335c/roles/8ccdb5d0cdab43deaae10d310e7578df` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.222534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb6407-fabf-48fe-a835-a192568291ce None admin] RBAC: Authorizing `identity:create_grant(project_id=5d7c3b139f284435aea4ff08a2f0952b, user_id=e628cb75e0c74f27a390945c8a04335c, role_id=8ccdb5d0cdab43deaae10d310e7578df)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.223967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb6407-fabf-48fe-a835-a192568291ce None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.234141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10789ea-25e8-45c8-a622-6f8212021549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.234973 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10789ea-25e8-45c8-a622-6f8212021549 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.235263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10789ea-25e8-45c8-a622-6f8212021549 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.235611 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10789ea-25e8-45c8-a622-6f8212021549 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.246990 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10789ea-25e8-45c8-a622-6f8212021549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-e10789ea-25e8-45c8-a622-6f8212021549 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:18.254666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:18.255789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:18.256679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:18.256679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:18.256679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:18.256679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:18.256679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:18.256679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:18.256679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:18.256679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:18.256679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:18.256679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:18.256679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:18.256679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:18.256679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 532/1079] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:44:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:18.261989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10789ea-25e8-45c8-a622-6f8212021549 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.323580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:18.356665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 548/1080] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:44:18 2022] PUT /identity/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/e628cb75e0c74f27a390945c8a04335c/roles/8ccdb5d0cdab43deaae10d310e7578df => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:44:18.368401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10789ea-25e8-45c8-a622-6f8212021549 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.408725 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3344759-6559-4534-bc72-6af9c261ebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.409427 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3344759-6559-4534-bc72-6af9c261ebd5 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.409714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3344759-6559-4534-bc72-6af9c261ebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.409992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3344759-6559-4534-bc72-6af9c261ebd5 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.410804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3344759-6559-4534-bc72-6af9c261ebd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:18.411117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3344759-6559-4534-bc72-6af9c261ebd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.412662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3344759-6559-4534-bc72-6af9c261ebd5 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.423904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 533/1081] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:44:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:18.430632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb6407-fabf-48fe-a835-a192568291ce None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.552045 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb6407-fabf-48fe-a835-a192568291ce None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.589132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d4d87-2535-418f-a018-9c72464fc81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.590041 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d4d87-2535-418f-a018-9c72464fc81b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.590394 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d4d87-2535-418f-a018-9c72464fc81b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.590636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d4d87-2535-418f-a018-9c72464fc81b None admin] PATH_INFO: `/v3/users/8ecfd254b29047a19f9b302c6e1c4f3f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.591868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d4d87-2535-418f-a018-9c72464fc81b None admin] RBAC: Authorizing `identity:get_user(user_id=8ecfd254b29047a19f9b302c6e1c4f3f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.593306 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d4d87-2535-418f-a018-9c72464fc81b None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.595196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 549/1082] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:18 2022] GET /identity/v3/users/8ecfd254b29047a19f9b302c6e1c4f3f => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:18.601078 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3344759-6559-4534-bc72-6af9c261ebd5 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.708119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3344759-6559-4534-bc72-6af9c261ebd5 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.730472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:18.746029 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634614af-a81b-44d5-8e0b-c0777241b5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.746940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634614af-a81b-44d5-8e0b-c0777241b5ef None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.747228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634614af-a81b-44d5-8e0b-c0777241b5ef None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.747511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634614af-a81b-44d5-8e0b-c0777241b5ef None admin] PATH_INFO: `/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.772304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634614af-a81b-44d5-8e0b-c0777241b5ef None admin] RBAC: Authorizing `identity:get_project(project_id=55a1b53bb39a4e6397e50ee9eff9802f)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.772304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634614af-a81b-44d5-8e0b-c0777241b5ef None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.772304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 534/1083] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:18 2022] GET /identity/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:18.772581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d4d87-2535-418f-a018-9c72464fc81b None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:18.789267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:18.863724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d4d87-2535-418f-a018-9c72464fc81b None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:18.901471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8ae94-e96d-4e53-a384-134695a89534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:18.902411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8ae94-e96d-4e53-a384-134695a89534 None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:18.902704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8ae94-e96d-4e53-a384-134695a89534 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:18.902991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8ae94-e96d-4e53-a384-134695a89534 None admin] PATH_INFO: `/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f/users/8ecfd254b29047a19f9b302c6e1c4f3f/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:18.905004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8ae94-e96d-4e53-a384-134695a89534 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a1b53bb39a4e6397e50ee9eff9802f, user_id=8ecfd254b29047a19f9b302c6e1c4f3f, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:18.906492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8ae94-e96d-4e53-a384-134695a89534 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:18.992677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 550/1084] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:44:18 2022] PUT /identity/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f/users/8ecfd254b29047a19f9b302c6e1c4f3f/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:44:20.403882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783418c-9e20-4be5-87bc-b98cc7e2e946 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:20.404456 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783418c-9e20-4be5-87bc-b98cc7e2e946 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:20.404456 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783418c-9e20-4be5-87bc-b98cc7e2e946 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:20.405822 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 535/1085] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:20.411603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a64116-d374-4272-a9b9-e763e5923be2 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:20.412089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a64116-d374-4272-a9b9-e763e5923be2 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:20.412089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a64116-d374-4272-a9b9-e763e5923be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:20.431225 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-50a64116-d374-4272-a9b9-e763e5923be2 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:20.432848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-50a64116-d374-4272-a9b9-e763e5923be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:20.513281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 551/1086] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:20.537136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783418c-9e20-4be5-87bc-b98cc7e2e946 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:20.659948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783418c-9e20-4be5-87bc-b98cc7e2e946 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:20.711030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17593c2-1bd1-4c45-94c6-42771feffc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:20.711869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17593c2-1bd1-4c45-94c6-42771feffc59 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:20.712151 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17593c2-1bd1-4c45-94c6-42771feffc59 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:20.712438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17593c2-1bd1-4c45-94c6-42771feffc59 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:20.730369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17593c2-1bd1-4c45-94c6-42771feffc59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-e17593c2-1bd1-4c45-94c6-42771feffc59 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:20.741645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:20.742809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:20.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:20.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:20.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:20.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:20.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:20.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:20.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:20.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:20.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:20.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:20.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 18:44:20.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:20.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 536/1087] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Jul 25 18:44:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:20.750587 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a64116-d374-4272-a9b9-e763e5923be2 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:20.772654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:20.879498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a64116-d374-4272-a9b9-e763e5923be2 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:20.925438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117d5d2-4edb-4ec1-8a6b-c30df8cc284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:20.926113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117d5d2-4edb-4ec1-8a6b-c30df8cc284b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:20.926427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117d5d2-4edb-4ec1-8a6b-c30df8cc284b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:20.926708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117d5d2-4edb-4ec1-8a6b-c30df8cc284b None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:20.927458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117d5d2-4edb-4ec1-8a6b-c30df8cc284b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:20.927770 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117d5d2-4edb-4ec1-8a6b-c30df8cc284b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:20.929132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117d5d2-4edb-4ec1-8a6b-c30df8cc284b None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:20.939380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 552/1088] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 18:44:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:20.945219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17593c2-1bd1-4c45-94c6-42771feffc59 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:20.997934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:21.032436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc5d81-323a-40a6-b806-33edcaa3f93d None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.032739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc5d81-323a-40a6-b806-33edcaa3f93d None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.033019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc5d81-323a-40a6-b806-33edcaa3f93d None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.034431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 553/1089] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:44:21.040016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f566-521c-42be-adbe-90a060b3c48e None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.040305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f566-521c-42be-adbe-90a060b3c48e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.040582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f566-521c-42be-adbe-90a060b3c48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.055305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17593c2-1bd1-4c45-94c6-42771feffc59 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.060210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-f9f4f566-521c-42be-adbe-90a060b3c48e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:21.061430 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f9f4f566-521c-42be-adbe-90a060b3c48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:21.092130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036edab2-0c63-4795-ba0d-23b8c0805419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.092927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036edab2-0c63-4795-ba0d-23b8c0805419 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.093225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036edab2-0c63-4795-ba0d-23b8c0805419 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.093495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036edab2-0c63-4795-ba0d-23b8c0805419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.105294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036edab2-0c63-4795-ba0d-23b8c0805419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.106789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036edab2-0c63-4795-ba0d-23b8c0805419 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-036edab2-0c63-4795-ba0d-23b8c0805419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:21.125325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:21.126315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:21.127485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:21.127485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:21.127485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:21.127485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:21.127485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 537/1090] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:21.133503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036edab2-0c63-4795-ba0d-23b8c0805419 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.138384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 554/1091] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:21.161810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f566-521c-42be-adbe-90a060b3c48e None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.174172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:21.239537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:21.242627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036edab2-0c63-4795-ba0d-23b8c0805419 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.271443 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f566-521c-42be-adbe-90a060b3c48e None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.280206 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55c849-8f53-4690-849e-a13264b75fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.280894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55c849-8f53-4690-849e-a13264b75fce None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.281192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55c849-8f53-4690-849e-a13264b75fce None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.281474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55c849-8f53-4690-849e-a13264b75fce None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.282213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55c849-8f53-4690-849e-a13264b75fce None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:21.282558 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55c849-8f53-4690-849e-a13264b75fce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.283934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55c849-8f53-4690-849e-a13264b75fce None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:21.295771 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 538/1092] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:21.301748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55c849-8f53-4690-849e-a13264b75fce None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.314759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a3bc0-5262-43f5-beba-b39bc281f4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.316085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a3bc0-5262-43f5-beba-b39bc281f4b3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.316578 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a3bc0-5262-43f5-beba-b39bc281f4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.317063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a3bc0-5262-43f5-beba-b39bc281f4b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.333050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a3bc0-5262-43f5-beba-b39bc281f4b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-077a3bc0-5262-43f5-beba-b39bc281f4b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:21.341669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:21.342755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:21.343599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:21.343599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:21.343599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:21.343599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:21.343599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:21.343599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:21.343599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:21.343599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:21.343599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:21.343599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:21.343599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:21.343599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:21.343599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 555/1093] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:21.349323 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a3bc0-5262-43f5-beba-b39bc281f4b3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.420804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55c849-8f53-4690-849e-a13264b75fce None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.434442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:21.460105 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e898b98-2bdc-4fcd-b448-e9dcc7155b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.460906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e898b98-2bdc-4fcd-b448-e9dcc7155b41 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.461198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e898b98-2bdc-4fcd-b448-e9dcc7155b41 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.461529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e898b98-2bdc-4fcd-b448-e9dcc7155b41 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.467211 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a3bc0-5262-43f5-beba-b39bc281f4b3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.469588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e898b98-2bdc-4fcd-b448-e9dcc7155b41 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.471107 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e898b98-2bdc-4fcd-b448-e9dcc7155b41 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-3e898b98-2bdc-4fcd-b448-e9dcc7155b41 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:44:21.487366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:21.488623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:21.489568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:21.489568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:21.489568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:44:21.489568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:44:21.489568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:44:21.489568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:44:21.489568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:44:21.489568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:44:21.489568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 18:44:21.489568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:21.489568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 539/1094] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:44:21.497511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e898b98-2bdc-4fcd-b448-e9dcc7155b41 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.511866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7e113-cbf7-4e35-88e3-dd93f89f9ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.512201 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7e113-cbf7-4e35-88e3-dd93f89f9ce8 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.512545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7e113-cbf7-4e35-88e3-dd93f89f9ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.512827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7e113-cbf7-4e35-88e3-dd93f89f9ce8 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.513679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7e113-cbf7-4e35-88e3-dd93f89f9ce8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:21.513984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7e113-cbf7-4e35-88e3-dd93f89f9ce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.515472 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7e113-cbf7-4e35-88e3-dd93f89f9ce8 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:21.524576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 556/1095] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:21.525612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:21.530718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7e113-cbf7-4e35-88e3-dd93f89f9ce8 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.611201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e898b98-2bdc-4fcd-b448-e9dcc7155b41 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.641562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7e113-cbf7-4e35-88e3-dd93f89f9ce8 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.653484 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879ab9e-f0a3-4e05-a78d-ed41d92f1a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.654319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879ab9e-f0a3-4e05-a78d-ed41d92f1a50 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.654788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879ab9e-f0a3-4e05-a78d-ed41d92f1a50 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.655086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879ab9e-f0a3-4e05-a78d-ed41d92f1a50 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.656244 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879ab9e-f0a3-4e05-a78d-ed41d92f1a50 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:21.656428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879ab9e-f0a3-4e05-a78d-ed41d92f1a50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.658008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879ab9e-f0a3-4e05-a78d-ed41d92f1a50 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:21.676087 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:21.677311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 540/1096] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:21.681361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f2f68-efa8-4e2e-98d5-63f1097c4d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.682497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f2f68-efa8-4e2e-98d5-63f1097c4d35 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.682929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f2f68-efa8-4e2e-98d5-63f1097c4d35 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.683112 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879ab9e-f0a3-4e05-a78d-ed41d92f1a50 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.683242 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f2f68-efa8-4e2e-98d5-63f1097c4d35 None admin] PATH_INFO: `/v3/users/8ecfd254b29047a19f9b302c6e1c4f3f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.684517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f2f68-efa8-4e2e-98d5-63f1097c4d35 None admin] RBAC: Authorizing `identity:get_user(user_id=8ecfd254b29047a19f9b302c6e1c4f3f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.692612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f2f68-efa8-4e2e-98d5-63f1097c4d35 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:21.692612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 557/1097] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/users/8ecfd254b29047a19f9b302c6e1c4f3f => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:21.694151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f2f68-efa8-4e2e-98d5-63f1097c4d35 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:21.779739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:21.804181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879ab9e-f0a3-4e05-a78d-ed41d92f1a50 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.805928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f2f68-efa8-4e2e-98d5-63f1097c4d35 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:21.843767 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba3e9b-06d3-40b6-b18a-daf268c522b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.844805 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba3e9b-06d3-40b6-b18a-daf268c522b6 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.845056 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd19476-606d-4d69-b024-40a5c6d1cbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:21.845305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba3e9b-06d3-40b6-b18a-daf268c522b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.845753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba3e9b-06d3-40b6-b18a-daf268c522b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.846161 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd19476-606d-4d69-b024-40a5c6d1cbc5 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:21.846712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd19476-606d-4d69-b024-40a5c6d1cbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:21.847121 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd19476-606d-4d69-b024-40a5c6d1cbc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:21.941911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:21.944187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba3e9b-06d3-40b6-b18a-daf268c522b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.945651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba3e9b-06d3-40b6-b18a-daf268c522b6 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:21.948323 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd19476-606d-4d69-b024-40a5c6d1cbc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:21.949789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd19476-606d-4d69-b024-40a5c6d1cbc5 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:22.033809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:22.057161 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 558/1098] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 18:44:22.057881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 541/1099] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:22.063484 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd19476-606d-4d69-b024-40a5c6d1cbc5 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:22.063643 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba3e9b-06d3-40b6-b18a-daf268c522b6 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:22.167828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:22.171540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba3e9b-06d3-40b6-b18a-daf268c522b6 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:22.173747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd19476-606d-4d69-b024-40a5c6d1cbc5 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:22.210786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d659ef-eeaf-4d26-90a4-c5e7358ca9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:22.211511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d659ef-eeaf-4d26-90a4-c5e7358ca9e3 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:22.211802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d659ef-eeaf-4d26-90a4-c5e7358ca9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:22.211932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7721c-bfb3-4b19-8485-3a13d49418e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:22.212070 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d659ef-eeaf-4d26-90a4-c5e7358ca9e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:22.212834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7721c-bfb3-4b19-8485-3a13d49418e8 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:22.213118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7721c-bfb3-4b19-8485-3a13d49418e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:22.213397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7721c-bfb3-4b19-8485-3a13d49418e8 None admin] PATH_INFO: `/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:22.215282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7721c-bfb3-4b19-8485-3a13d49418e8 None admin] RBAC: Authorizing `identity:get_project(project_id=55a1b53bb39a4e6397e50ee9eff9802f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:22.216727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7721c-bfb3-4b19-8485-3a13d49418e8 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:22.218576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 559/1100] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:22 2022] GET /identity/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:22.223270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d659ef-eeaf-4d26-90a4-c5e7358ca9e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:22.224646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7721c-bfb3-4b19-8485-3a13d49418e8 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:22.224776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d659ef-eeaf-4d26-90a4-c5e7358ca9e3 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-29d659ef-eeaf-4d26-90a4-c5e7358ca9e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:22.244592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:22.245700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:22.246816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:44:22.246816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:22.246816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:44:22.246816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:22.246816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 542/1101] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:22.252791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d659ef-eeaf-4d26-90a4-c5e7358ca9e3 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:22.285639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:22.335150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7721c-bfb3-4b19-8485-3a13d49418e8 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:22.363437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d659ef-eeaf-4d26-90a4-c5e7358ca9e3 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:22.377114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5f2c4-7cf3-4423-8d43-367efd45e4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:22.377849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5f2c4-7cf3-4423-8d43-367efd45e4db None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:22.378138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5f2c4-7cf3-4423-8d43-367efd45e4db None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:22.378453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5f2c4-7cf3-4423-8d43-367efd45e4db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:22.379295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5f2c4-7cf3-4423-8d43-367efd45e4db None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=55a1b53bb39a4e6397e50ee9eff9802f, user.id=8ecfd254b29047a19f9b302c6e1c4f3f) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:22.379609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5f2c4-7cf3-4423-8d43-367efd45e4db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:22.381062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5f2c4-7cf3-4423-8d43-367efd45e4db None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:22.388093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 560/1102] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:22 2022] GET /identity/v3/role_assignments?user.id=8ecfd254b29047a19f9b302c6e1c4f3f&scope.project.id=55a1b53bb39a4e6397e50ee9eff9802f&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:22.405059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e53209-c9c1-49b9-9522-ce1e65dbac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:22.405765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e53209-c9c1-49b9-9522-ce1e65dbac7c None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:22.406053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e53209-c9c1-49b9-9522-ce1e65dbac7c None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:22.406358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e53209-c9c1-49b9-9522-ce1e65dbac7c None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:22.410515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e53209-c9c1-49b9-9522-ce1e65dbac7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:22.410515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e53209-c9c1-49b9-9522-ce1e65dbac7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:22.410515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e53209-c9c1-49b9-9522-ce1e65dbac7c None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:22.423113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 543/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:22.430120 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5f2c4-7cf3-4423-8d43-367efd45e4db None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:22.469400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:22.551279 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5f2c4-7cf3-4423-8d43-367efd45e4db None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:22.591009 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b32842-b69a-4247-b01a-3e22600e4458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:22.591819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b32842-b69a-4247-b01a-3e22600e4458 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:22.592107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b32842-b69a-4247-b01a-3e22600e4458 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:22.592412 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b32842-b69a-4247-b01a-3e22600e4458 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:22.600579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b32842-b69a-4247-b01a-3e22600e4458 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:22.602056 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b32842-b69a-4247-b01a-3e22600e4458 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-83b32842-b69a-4247-b01a-3e22600e4458 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:44:22.617118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:22.618131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:22.619124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:44:22.619124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:44:22.619124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:44:22.619124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:44:22.619124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:44:22.619124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:44:22.619124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:44:22.619124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:22.619124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 561/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:44:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:22.629446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e53209-c9c1-49b9-9522-ce1e65dbac7c None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:22.729996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:22.740075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e53209-c9c1-49b9-9522-ce1e65dbac7c None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:22.779257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc17e8-91bf-41b2-92e7-027ff37e19e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:22.779930 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc17e8-91bf-41b2-92e7-027ff37e19e3 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:22.780240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc17e8-91bf-41b2-92e7-027ff37e19e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:22.780520 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc17e8-91bf-41b2-92e7-027ff37e19e3 None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:22.781316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc17e8-91bf-41b2-92e7-027ff37e19e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:22.781625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc17e8-91bf-41b2-92e7-027ff37e19e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:22.783191 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc17e8-91bf-41b2-92e7-027ff37e19e3 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:22.795983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 544/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:44:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:22.802621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b32842-b69a-4247-b01a-3e22600e4458 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:22.901597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:22.910688 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b32842-b69a-4247-b01a-3e22600e4458 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:22.949075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b5840-7197-4e02-836f-0d73c7141122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:22.949750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5840-7197-4e02-836f-0d73c7141122 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:22.950072 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5840-7197-4e02-836f-0d73c7141122 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:22.950375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5840-7197-4e02-836f-0d73c7141122 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:22.951222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b5840-7197-4e02-836f-0d73c7141122 None admin] RBAC: Adding query filter params (role.id=8ccdb5d0cdab43deaae10d310e7578df, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=e628cb75e0c74f27a390945c8a04335c) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:22.951544 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b5840-7197-4e02-836f-0d73c7141122 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:22.952982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b5840-7197-4e02-836f-0d73c7141122 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:22.959685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 562/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:22 2022] GET /identity/v3/role_assignments?user.id=e628cb75e0c74f27a390945c8a04335c&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=8ccdb5d0cdab43deaae10d310e7578df => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:24.503216 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d67bf-7ed5-4d42-878a-b6fda12c0783 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:24.503757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d67bf-7ed5-4d42-878a-b6fda12c0783 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:24.503757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d67bf-7ed5-4d42-878a-b6fda12c0783 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:24.505308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 545/1107] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:24.510716 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcff0ad-e93f-49f5-91e8-825c86399cb9 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:24.511165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcff0ad-e93f-49f5-91e8-825c86399cb9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:24.511291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcff0ad-e93f-49f5-91e8-825c86399cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:24.530215 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ddcff0ad-e93f-49f5-91e8-825c86399cb9 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:24.531465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ddcff0ad-e93f-49f5-91e8-825c86399cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:24.600602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 563/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:24.625770 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d67bf-7ed5-4d42-878a-b6fda12c0783 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:24.731630 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d67bf-7ed5-4d42-878a-b6fda12c0783 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:24.766372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155dfc5-567c-4e54-9573-3bb22c08908a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:24.767101 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155dfc5-567c-4e54-9573-3bb22c08908a None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:24.767389 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155dfc5-567c-4e54-9573-3bb22c08908a None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:24.767675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155dfc5-567c-4e54-9573-3bb22c08908a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:24.768854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155dfc5-567c-4e54-9573-3bb22c08908a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:24.770276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155dfc5-567c-4e54-9573-3bb22c08908a None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:24.771936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 546/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:24.777851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcff0ad-e93f-49f5-91e8-825c86399cb9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:24.866122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:24.881799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcff0ad-e93f-49f5-91e8-825c86399cb9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:24.916854 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc11f9-3c2e-4407-8707-2ac03fc86cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:24.917535 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc11f9-3c2e-4407-8707-2ac03fc86cab None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:24.917826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc11f9-3c2e-4407-8707-2ac03fc86cab None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:24.918110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc11f9-3c2e-4407-8707-2ac03fc86cab None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:24.918848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc11f9-3c2e-4407-8707-2ac03fc86cab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:24.920295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc11f9-3c2e-4407-8707-2ac03fc86cab None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:25.032689 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 564/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:44:24 2022] POST /identity/v3/users => generated 286 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:27.130489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b9288-0ac2-4542-975b-3d2ae221dbda None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:27.131027 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b9288-0ac2-4542-975b-3d2ae221dbda None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:27.131027 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b9288-0ac2-4542-975b-3d2ae221dbda None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:27.132675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 547/1111] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:27.138547 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dba5f-bf25-4e35-b09a-93e920ef8281 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:27.139007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dba5f-bf25-4e35-b09a-93e920ef8281 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:27.139117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dba5f-bf25-4e35-b09a-93e920ef8281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:27.157705 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-403dba5f-bf25-4e35-b09a-93e920ef8281 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:27.158954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-403dba5f-bf25-4e35-b09a-93e920ef8281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:27.230307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 565/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:27.254962 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b9288-0ac2-4542-975b-3d2ae221dbda None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:27.274954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:27.363100 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b9288-0ac2-4542-975b-3d2ae221dbda None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:27.403879 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7810ce-7808-4473-ad5a-f8308cfa25e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:27.404346 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7810ce-7808-4473-ad5a-f8308cfa25e6 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:27.404636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7810ce-7808-4473-ad5a-f8308cfa25e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:27.404918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7810ce-7808-4473-ad5a-f8308cfa25e6 None admin] PATH_INFO: `/v3/roles/a638e91b70624f669ffe76dac32323f5` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:27.406518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7810ce-7808-4473-ad5a-f8308cfa25e6 None admin] RBAC: Authorizing `identity:get_role(role_id=a638e91b70624f669ffe76dac32323f5)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:27.407959 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7810ce-7808-4473-ad5a-f8308cfa25e6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:27.409378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 548/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:27 2022] GET /identity/v3/roles/a638e91b70624f669ffe76dac32323f5 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:27.417195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403dba5f-bf25-4e35-b09a-93e920ef8281 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:27.503200 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:27.533801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403dba5f-bf25-4e35-b09a-93e920ef8281 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:27.568322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab1a8f-787c-4108-8d55-9e97383c1818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:27.569221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab1a8f-787c-4108-8d55-9e97383c1818 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:27.569470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab1a8f-787c-4108-8d55-9e97383c1818 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:27.569741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab1a8f-787c-4108-8d55-9e97383c1818 None admin] PATH_INFO: `/v3/users/da6aca5d4fd54917b59bec315a3e3c2a` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:27.584691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab1a8f-787c-4108-8d55-9e97383c1818 None admin] RBAC: Authorizing `identity:get_user(user_id=da6aca5d4fd54917b59bec315a3e3c2a)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:27.586146 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab1a8f-787c-4108-8d55-9e97383c1818 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:27.588020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 566/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:27 2022] GET /identity/v3/users/da6aca5d4fd54917b59bec315a3e3c2a => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:27.594474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7810ce-7808-4473-ad5a-f8308cfa25e6 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:27.689892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:27.705020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7810ce-7808-4473-ad5a-f8308cfa25e6 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:27.742600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03a703-e8cd-4612-a8f1-42474e352cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:27.743293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03a703-e8cd-4612-a8f1-42474e352cec None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:27.743627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03a703-e8cd-4612-a8f1-42474e352cec None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:27.743907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03a703-e8cd-4612-a8f1-42474e352cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:27.826597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03a703-e8cd-4612-a8f1-42474e352cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:27.828048 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03a703-e8cd-4612-a8f1-42474e352cec None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:27.911445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:27.927144 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 549/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:27.934138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab1a8f-787c-4108-8d55-9e97383c1818 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:28.034660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab1a8f-787c-4108-8d55-9e97383c1818 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:28.066691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:28.072865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88919d2a-a873-489d-b93c-1278afa268bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:28.073829 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88919d2a-a873-489d-b93c-1278afa268bc None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:28.074118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88919d2a-a873-489d-b93c-1278afa268bc None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:28.074436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88919d2a-a873-489d-b93c-1278afa268bc None admin] PATH_INFO: `/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:28.075632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88919d2a-a873-489d-b93c-1278afa268bc None admin] RBAC: Authorizing `identity:get_project(project_id=55a1b53bb39a4e6397e50ee9eff9802f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:28.077071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88919d2a-a873-489d-b93c-1278afa268bc None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:28.078988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 567/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:27 2022] GET /identity/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:28.085193 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03a703-e8cd-4612-a8f1-42474e352cec None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:28.188600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03a703-e8cd-4612-a8f1-42474e352cec None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:28.224854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21644ea-4e1d-4f2a-a88f-d3b569370642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:28.225514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21644ea-4e1d-4f2a-a88f-d3b569370642 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:28.225804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21644ea-4e1d-4f2a-a88f-d3b569370642 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:28.226172 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21644ea-4e1d-4f2a-a88f-d3b569370642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:28.227051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21644ea-4e1d-4f2a-a88f-d3b569370642 None admin] RBAC: Adding query filter params (role.id=a638e91b70624f669ffe76dac32323f5, scope.project.id=55a1b53bb39a4e6397e50ee9eff9802f, user.id=da6aca5d4fd54917b59bec315a3e3c2a) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:28.227378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21644ea-4e1d-4f2a-a88f-d3b569370642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:28.229676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21644ea-4e1d-4f2a-a88f-d3b569370642 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:28.235517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 550/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:28 2022] GET /identity/v3/role_assignments?user.id=da6aca5d4fd54917b59bec315a3e3c2a&scope.project.id=55a1b53bb39a4e6397e50ee9eff9802f&role.id=a638e91b70624f669ffe76dac32323f5 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:30.324861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408cec6-7a2a-4c4c-b675-88446df6f1a8 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:30.325462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408cec6-7a2a-4c4c-b675-88446df6f1a8 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:30.325462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408cec6-7a2a-4c4c-b675-88446df6f1a8 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:30.326819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 568/1118] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:30.332124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9680b-546e-46c5-91df-c90ed49cec61 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:30.332595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9680b-546e-46c5-91df-c90ed49cec61 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:30.332700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9680b-546e-46c5-91df-c90ed49cec61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:30.350965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-5ed9680b-546e-46c5-91df-c90ed49cec61 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:30.352173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5ed9680b-546e-46c5-91df-c90ed49cec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:30.401877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:30.423340 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 551/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:30.445987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8408cec6-7a2a-4c4c-b675-88446df6f1a8 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:30.549642 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8408cec6-7a2a-4c4c-b675-88446df6f1a8 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:30.583745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f48f96-9975-4ed2-89b8-36388a53cf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:30.584608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f48f96-9975-4ed2-89b8-36388a53cf1f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:30.584891 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f48f96-9975-4ed2-89b8-36388a53cf1f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:30.585165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f48f96-9975-4ed2-89b8-36388a53cf1f None admin] PATH_INFO: `/v3/roles/a638e91b70624f669ffe76dac32323f5` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:30.586684 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f48f96-9975-4ed2-89b8-36388a53cf1f None admin] RBAC: Authorizing `identity:get_role(role_id=a638e91b70624f669ffe76dac32323f5)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:30.588077 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f48f96-9975-4ed2-89b8-36388a53cf1f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:30.589454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 569/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:30 2022] GET /identity/v3/roles/a638e91b70624f669ffe76dac32323f5 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:30.595561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9680b-546e-46c5-91df-c90ed49cec61 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:30.699680 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9680b-546e-46c5-91df-c90ed49cec61 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:30.734839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861e05c-1b2d-41c4-ba76-a1664bc50030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:30.735790 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861e05c-1b2d-41c4-ba76-a1664bc50030 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:30.736081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861e05c-1b2d-41c4-ba76-a1664bc50030 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:30.736364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861e05c-1b2d-41c4-ba76-a1664bc50030 None admin] PATH_INFO: `/v3/users/da6aca5d4fd54917b59bec315a3e3c2a` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:30.737575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861e05c-1b2d-41c4-ba76-a1664bc50030 None admin] RBAC: Authorizing `identity:get_user(user_id=da6aca5d4fd54917b59bec315a3e3c2a)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:30.739115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861e05c-1b2d-41c4-ba76-a1664bc50030 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:30.740842 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 552/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:30 2022] GET /identity/v3/users/da6aca5d4fd54917b59bec315a3e3c2a => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:30.747645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f48f96-9975-4ed2-89b8-36388a53cf1f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:30.815703 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:30.855356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f48f96-9975-4ed2-89b8-36388a53cf1f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:30.890737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08369f-9c81-46c1-b537-d08127c97dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:30.891659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08369f-9c81-46c1-b537-d08127c97dd4 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:30.892030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08369f-9c81-46c1-b537-d08127c97dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:30.892323 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08369f-9c81-46c1-b537-d08127c97dd4 None admin] PATH_INFO: `/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:30.893538 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08369f-9c81-46c1-b537-d08127c97dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=55a1b53bb39a4e6397e50ee9eff9802f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:30.895012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08369f-9c81-46c1-b537-d08127c97dd4 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:30.896727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 570/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:30 2022] GET /identity/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:30.903101 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861e05c-1b2d-41c4-ba76-a1664bc50030 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:30.953963 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:31.006828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861e05c-1b2d-41c4-ba76-a1664bc50030 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:31.042552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38315f-a0cd-46e8-803e-9257c7ccd72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:31.043512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38315f-a0cd-46e8-803e-9257c7ccd72f None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:31.043801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38315f-a0cd-46e8-803e-9257c7ccd72f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:31.044082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38315f-a0cd-46e8-803e-9257c7ccd72f None admin] PATH_INFO: `/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f/users/da6aca5d4fd54917b59bec315a3e3c2a/roles/a638e91b70624f669ffe76dac32323f5` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:31.046354 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38315f-a0cd-46e8-803e-9257c7ccd72f None admin] RBAC: Authorizing `identity:create_grant(project_id=55a1b53bb39a4e6397e50ee9eff9802f, user_id=da6aca5d4fd54917b59bec315a3e3c2a, role_id=a638e91b70624f669ffe76dac32323f5)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:31.047902 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38315f-a0cd-46e8-803e-9257c7ccd72f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:31.221710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 553/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:44:30 2022] PUT /identity/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f/users/da6aca5d4fd54917b59bec315a3e3c2a/roles/a638e91b70624f669ffe76dac32323f5 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:44:33.279546 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f42e79-1401-4c11-bfa2-563cfceebd7f None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:33.280120 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f42e79-1401-4c11-bfa2-563cfceebd7f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:33.280120 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f42e79-1401-4c11-bfa2-563cfceebd7f None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:33.281522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 571/1124] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:33.286917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c4807-0691-4ebe-a8d5-fd27c2b766b2 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:33.287335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c4807-0691-4ebe-a8d5-fd27c2b766b2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:33.287485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c4807-0691-4ebe-a8d5-fd27c2b766b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:33.306258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3e5c4807-0691-4ebe-a8d5-fd27c2b766b2 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:33.308030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3e5c4807-0691-4ebe-a8d5-fd27c2b766b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:33.377158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 554/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:33.404273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f42e79-1401-4c11-bfa2-563cfceebd7f None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:33.517297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f42e79-1401-4c11-bfa2-563cfceebd7f None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:33.551940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cef0b7-5804-4b4e-a6f0-7718bbd7a919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:33.552799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cef0b7-5804-4b4e-a6f0-7718bbd7a919 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:33.553086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cef0b7-5804-4b4e-a6f0-7718bbd7a919 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:33.553364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cef0b7-5804-4b4e-a6f0-7718bbd7a919 None admin] PATH_INFO: `/v3/roles/a638e91b70624f669ffe76dac32323f5` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:33.554956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cef0b7-5804-4b4e-a6f0-7718bbd7a919 None admin] RBAC: Authorizing `identity:get_role(role_id=a638e91b70624f669ffe76dac32323f5)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:33.556333 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cef0b7-5804-4b4e-a6f0-7718bbd7a919 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:33.558508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 572/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:33 2022] GET /identity/v3/roles/a638e91b70624f669ffe76dac32323f5 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:33.564653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c4807-0691-4ebe-a8d5-fd27c2b766b2 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:33.605397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:33.669798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c4807-0691-4ebe-a8d5-fd27c2b766b2 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:33.705855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc8777-6041-4110-84b1-63f09faedd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:33.706781 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc8777-6041-4110-84b1-63f09faedd1b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:33.707055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc8777-6041-4110-84b1-63f09faedd1b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:33.707335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc8777-6041-4110-84b1-63f09faedd1b None admin] PATH_INFO: `/v3/users/da6aca5d4fd54917b59bec315a3e3c2a` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:33.708527 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc8777-6041-4110-84b1-63f09faedd1b None admin] RBAC: Authorizing `identity:get_user(user_id=da6aca5d4fd54917b59bec315a3e3c2a)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:33.709960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc8777-6041-4110-84b1-63f09faedd1b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:33.711645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 555/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:33 2022] GET /identity/v3/users/da6aca5d4fd54917b59bec315a3e3c2a => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:33.717620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cef0b7-5804-4b4e-a6f0-7718bbd7a919 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:33.720675 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:33.822185 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cef0b7-5804-4b4e-a6f0-7718bbd7a919 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:33.856885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00312750-5479-4e8b-9ba9-5d7b03c7caef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:33.857607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00312750-5479-4e8b-9ba9-5d7b03c7caef None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:33.857898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00312750-5479-4e8b-9ba9-5d7b03c7caef None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:33.858172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00312750-5479-4e8b-9ba9-5d7b03c7caef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:33.926417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00312750-5479-4e8b-9ba9-5d7b03c7caef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:33.927878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00312750-5479-4e8b-9ba9-5d7b03c7caef None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:33.929831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:34.030165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 573/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:34.036229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc8777-6041-4110-84b1-63f09faedd1b None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:34.146945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc8777-6041-4110-84b1-63f09faedd1b None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:34.156591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:34.185666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb3b1b-3760-4571-9a6b-25440da80ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:34.186574 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb3b1b-3760-4571-9a6b-25440da80ecd None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:34.186861 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb3b1b-3760-4571-9a6b-25440da80ecd None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:34.187141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb3b1b-3760-4571-9a6b-25440da80ecd None admin] PATH_INFO: `/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:34.188433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb3b1b-3760-4571-9a6b-25440da80ecd None admin] RBAC: Authorizing `identity:get_project(project_id=55a1b53bb39a4e6397e50ee9eff9802f)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:34.189889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb3b1b-3760-4571-9a6b-25440da80ecd None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:34.191743 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 556/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:34 2022] GET /identity/v3/projects/55a1b53bb39a4e6397e50ee9eff9802f => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:34.198383 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00312750-5479-4e8b-9ba9-5d7b03c7caef None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:34.302685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00312750-5479-4e8b-9ba9-5d7b03c7caef None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:34.304502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:34.340882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4949f-84f0-4f6d-885a-2ae03a6f42cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:34.341628 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4949f-84f0-4f6d-885a-2ae03a6f42cd None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:34.341864 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4949f-84f0-4f6d-885a-2ae03a6f42cd None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:34.344439 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4949f-84f0-4f6d-885a-2ae03a6f42cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:34.344439 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4949f-84f0-4f6d-885a-2ae03a6f42cd None admin] RBAC: Adding query filter params (role.id=a638e91b70624f669ffe76dac32323f5, scope.project.id=55a1b53bb39a4e6397e50ee9eff9802f, user.id=da6aca5d4fd54917b59bec315a3e3c2a) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:34.344439 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4949f-84f0-4f6d-885a-2ae03a6f42cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:34.344708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4949f-84f0-4f6d-885a-2ae03a6f42cd None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:34.351773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 574/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:34 2022] GET /identity/v3/role_assignments?user.id=da6aca5d4fd54917b59bec315a3e3c2a&scope.project.id=55a1b53bb39a4e6397e50ee9eff9802f&role.id=a638e91b70624f669ffe76dac32323f5 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:36.418279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d86b4-0c29-4431-81ac-26f6d1fc5377 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:36.418877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d86b4-0c29-4431-81ac-26f6d1fc5377 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:36.418877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d86b4-0c29-4431-81ac-26f6d1fc5377 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:36.420148 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 557/1131] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:36.425577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252bd1f-4ef9-45e5-9a2b-5b5981a5d3f0 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:36.426049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252bd1f-4ef9-45e5-9a2b-5b5981a5d3f0 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:36.426143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252bd1f-4ef9-45e5-9a2b-5b5981a5d3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:36.445090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-9252bd1f-4ef9-45e5-9a2b-5b5981a5d3f0 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:36.446294 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9252bd1f-4ef9-45e5-9a2b-5b5981a5d3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:36.520293 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 575/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:36.543588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d86b4-0c29-4431-81ac-26f6d1fc5377 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:36.650004 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d86b4-0c29-4431-81ac-26f6d1fc5377 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:36.685087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00183f3-a9f9-46bf-96f6-119cc2ca3f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:36.685888 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00183f3-a9f9-46bf-96f6-119cc2ca3f13 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:36.686168 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00183f3-a9f9-46bf-96f6-119cc2ca3f13 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:36.686469 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00183f3-a9f9-46bf-96f6-119cc2ca3f13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:36.687650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00183f3-a9f9-46bf-96f6-119cc2ca3f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:36.689085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00183f3-a9f9-46bf-96f6-119cc2ca3f13 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:36.690757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 558/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:36.696907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252bd1f-4ef9-45e5-9a2b-5b5981a5d3f0 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:36.793213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:36.799094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252bd1f-4ef9-45e5-9a2b-5b5981a5d3f0 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:36.833754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a254a-8dd8-436c-9cc9-7d4fe0f5c7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:36.834456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a254a-8dd8-436c-9cc9-7d4fe0f5c7eb None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:36.834741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a254a-8dd8-436c-9cc9-7d4fe0f5c7eb None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:36.835017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a254a-8dd8-436c-9cc9-7d4fe0f5c7eb None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:36.835730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a254a-8dd8-436c-9cc9-7d4fe0f5c7eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:36.837156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a254a-8dd8-436c-9cc9-7d4fe0f5c7eb None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:36.988157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 576/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:44:36 2022] POST /identity/v3/projects => generated 311 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:39.090250 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108475a-48db-48b5-a391-ebcf32564c77 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:39.090849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108475a-48db-48b5-a391-ebcf32564c77 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:39.090849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108475a-48db-48b5-a391-ebcf32564c77 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:39.092223 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 559/1135] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:39.097613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c45e6-4170-4b34-9c15-9a6952112242 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:39.098264 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c45e6-4170-4b34-9c15-9a6952112242 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:39.098264 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c45e6-4170-4b34-9c15-9a6952112242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:39.117765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-094c45e6-4170-4b34-9c15-9a6952112242 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:39.119015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-094c45e6-4170-4b34-9c15-9a6952112242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:39.190121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 577/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:39.215192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108475a-48db-48b5-a391-ebcf32564c77 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:39.262444 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:39.323371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108475a-48db-48b5-a391-ebcf32564c77 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:39.359870 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fde7e-fe83-456c-9a1d-cb7125d74467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:39.360738 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fde7e-fe83-456c-9a1d-cb7125d74467 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:39.361027 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fde7e-fe83-456c-9a1d-cb7125d74467 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:39.361314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fde7e-fe83-456c-9a1d-cb7125d74467 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:39.362534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fde7e-fe83-456c-9a1d-cb7125d74467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:39.363970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fde7e-fe83-456c-9a1d-cb7125d74467 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:39.365602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 560/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:44:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:39.372104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c45e6-4170-4b34-9c15-9a6952112242 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:39.454658 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:39.476666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c45e6-4170-4b34-9c15-9a6952112242 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:39.512117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66dd66-0d50-4110-a475-fc96780164d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:39.512845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66dd66-0d50-4110-a475-fc96780164d9 None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:39.513136 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66dd66-0d50-4110-a475-fc96780164d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:39.513419 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66dd66-0d50-4110-a475-fc96780164d9 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:39.514133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66dd66-0d50-4110-a475-fc96780164d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:39.515599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66dd66-0d50-4110-a475-fc96780164d9 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:39.710748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 578/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:44:39 2022] POST /identity/v3/users => generated 286 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:41.790730 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d83560-258f-4375-87fc-b00bf1c7e973 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:41.791271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d83560-258f-4375-87fc-b00bf1c7e973 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:41.791271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d83560-258f-4375-87fc-b00bf1c7e973 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:41.793321 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 561/1139] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:41.798738 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e15625-2db4-43d8-ae8a-6baa82c41010 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:41.799212 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e15625-2db4-43d8-ae8a-6baa82c41010 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:41.799306 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e15625-2db4-43d8-ae8a-6baa82c41010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:41.818464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-c5e15625-2db4-43d8-ae8a-6baa82c41010 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:41.819680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c5e15625-2db4-43d8-ae8a-6baa82c41010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:41.890051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 579/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:41.913382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d83560-258f-4375-87fc-b00bf1c7e973 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:42.030227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d83560-258f-4375-87fc-b00bf1c7e973 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:42.041305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:42.065490 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5c534-5414-4837-9297-0b734b997ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:42.066280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5c534-5414-4837-9297-0b734b997ea5 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:42.066599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5c534-5414-4837-9297-0b734b997ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:42.066876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5c534-5414-4837-9297-0b734b997ea5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:42.078303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5c534-5414-4837-9297-0b734b997ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-b0b5c534-5414-4837-9297-0b734b997ea5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:42.085936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:42.087246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:42.088451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:42.088451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:42.088451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:42.088451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:42.088451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:42.088451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:42.088451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:42.088451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:42.088451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:42.088451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:42.088451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:42.088451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:42.088451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 562/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:44:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:42.093873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e15625-2db4-43d8-ae8a-6baa82c41010 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:42.158058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:42.198175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e15625-2db4-43d8-ae8a-6baa82c41010 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:42.241135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdad44-cad5-49bd-ba20-6b7687ce6e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:42.241807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdad44-cad5-49bd-ba20-6b7687ce6e7f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:42.242101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdad44-cad5-49bd-ba20-6b7687ce6e7f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:42.242412 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdad44-cad5-49bd-ba20-6b7687ce6e7f None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:42.243225 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cdad44-cad5-49bd-ba20-6b7687ce6e7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:42.243537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cdad44-cad5-49bd-ba20-6b7687ce6e7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:42.245516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cdad44-cad5-49bd-ba20-6b7687ce6e7f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:42.254157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 580/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:44:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:44:42.260367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5c534-5414-4837-9297-0b734b997ea5 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:42.365349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5c534-5414-4837-9297-0b734b997ea5 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:42.460285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c0dff-95d2-4db3-aaf9-a94229453dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:42.461193 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c0dff-95d2-4db3-aaf9-a94229453dd0 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:42.461484 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c0dff-95d2-4db3-aaf9-a94229453dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:42.461763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c0dff-95d2-4db3-aaf9-a94229453dd0 None admin] PATH_INFO: `/v3/users/6de75ba5114649f389a53109323096dd` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:42.476043 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c0dff-95d2-4db3-aaf9-a94229453dd0 None admin] RBAC: Authorizing `identity:get_user(user_id=6de75ba5114649f389a53109323096dd)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:42.477510 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c0dff-95d2-4db3-aaf9-a94229453dd0 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:42.479603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 563/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:42 2022] GET /identity/v3/users/6de75ba5114649f389a53109323096dd => generated 286 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:42.485718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdad44-cad5-49bd-ba20-6b7687ce6e7f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:42.591310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdad44-cad5-49bd-ba20-6b7687ce6e7f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:42.617257 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:42.626858 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf12b5-8920-4310-9e5d-c4d17b14c162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:42.627536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf12b5-8920-4310-9e5d-c4d17b14c162 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:42.627826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf12b5-8920-4310-9e5d-c4d17b14c162 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:42.628106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf12b5-8920-4310-9e5d-c4d17b14c162 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:42.699089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf12b5-8920-4310-9e5d-c4d17b14c162 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:42.700460 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf12b5-8920-4310-9e5d-c4d17b14c162 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:42.808430 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 581/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:42.814466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c0dff-95d2-4db3-aaf9-a94229453dd0 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:42.844364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:42.919445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c0dff-95d2-4db3-aaf9-a94229453dd0 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:42.953934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b33ce-5fa7-4a99-964d-46b368c95d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:42.954820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b33ce-5fa7-4a99-964d-46b368c95d61 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:42.955111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b33ce-5fa7-4a99-964d-46b368c95d61 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:42.955394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b33ce-5fa7-4a99-964d-46b368c95d61 None admin] PATH_INFO: `/v3/projects/41a7bff642274a40b61c5b161277b623` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:42.956656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b33ce-5fa7-4a99-964d-46b368c95d61 None admin] RBAC: Authorizing `identity:get_project(project_id=41a7bff642274a40b61c5b161277b623)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:42.958017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b33ce-5fa7-4a99-964d-46b368c95d61 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:42.959781 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 564/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:42 2022] GET /identity/v3/projects/41a7bff642274a40b61c5b161277b623 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:42.966588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf12b5-8920-4310-9e5d-c4d17b14c162 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:42.993966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:43.068592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf12b5-8920-4310-9e5d-c4d17b14c162 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:43.111344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93689ecc-55b0-4bb3-9da8-d9dc0a0c442b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:43.112056 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93689ecc-55b0-4bb3-9da8-d9dc0a0c442b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:43.112351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93689ecc-55b0-4bb3-9da8-d9dc0a0c442b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:43.112634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93689ecc-55b0-4bb3-9da8-d9dc0a0c442b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:43.113465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93689ecc-55b0-4bb3-9da8-d9dc0a0c442b None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=41a7bff642274a40b61c5b161277b623, user.id=6de75ba5114649f389a53109323096dd) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:43.113786 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93689ecc-55b0-4bb3-9da8-d9dc0a0c442b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:43.115248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93689ecc-55b0-4bb3-9da8-d9dc0a0c442b None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:43.121770 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 582/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:42 2022] GET /identity/v3/role_assignments?user.id=6de75ba5114649f389a53109323096dd&scope.project.id=41a7bff642274a40b61c5b161277b623&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:45.159647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd159c-e4be-49d8-a8bb-75f0ff650009 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:45.160022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd159c-e4be-49d8-a8bb-75f0ff650009 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:45.160022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd159c-e4be-49d8-a8bb-75f0ff650009 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:45.161421 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 565/1147] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:45.167375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c33e3-e3cb-4e35-ae88-112efbd040b1 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:45.168040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c33e3-e3cb-4e35-ae88-112efbd040b1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:45.168040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c33e3-e3cb-4e35-ae88-112efbd040b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:45.186519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-a24c33e3-e3cb-4e35-ae88-112efbd040b1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:45.187731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a24c33e3-e3cb-4e35-ae88-112efbd040b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:45.257389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 583/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:45.281408 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd159c-e4be-49d8-a8bb-75f0ff650009 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:45.385095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:45.392994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd159c-e4be-49d8-a8bb-75f0ff650009 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:45.428121 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb75a7-2c18-42d0-90d2-f435f849b8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:45.428763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb75a7-2c18-42d0-90d2-f435f849b8f5 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:45.429054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb75a7-2c18-42d0-90d2-f435f849b8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:45.429331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb75a7-2c18-42d0-90d2-f435f849b8f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:45.440315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb75a7-2c18-42d0-90d2-f435f849b8f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-8cbb75a7-2c18-42d0-90d2-f435f849b8f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:45.447966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:45.449640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:45.450876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:45.450876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:45.450876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:45.450876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:45.450876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:45.450876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:45.450876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:45.450876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:45.450876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:45.450876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:45.450876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:45.450876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:45.450876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 566/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:44:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:45.456458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c33e3-e3cb-4e35-ae88-112efbd040b1 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:45.478952 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:45.561948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c33e3-e3cb-4e35-ae88-112efbd040b1 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:45.598489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c5254-62f0-4ff8-afa6-6d75857e4aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:45.599159 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c5254-62f0-4ff8-afa6-6d75857e4aae None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:45.599443 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c5254-62f0-4ff8-afa6-6d75857e4aae None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:45.599718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c5254-62f0-4ff8-afa6-6d75857e4aae None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:45.600458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c5254-62f0-4ff8-afa6-6d75857e4aae None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:45.600773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c5254-62f0-4ff8-afa6-6d75857e4aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:45.602380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c5254-62f0-4ff8-afa6-6d75857e4aae None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:45.611773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 584/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:44:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:45.618018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb75a7-2c18-42d0-90d2-f435f849b8f5 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:45.723413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb75a7-2c18-42d0-90d2-f435f849b8f5 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:45.759740 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860c11d-c4e6-4b29-98ff-b345fa7d433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:45.760722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860c11d-c4e6-4b29-98ff-b345fa7d433a None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:45.761005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860c11d-c4e6-4b29-98ff-b345fa7d433a None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:45.761285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860c11d-c4e6-4b29-98ff-b345fa7d433a None admin] PATH_INFO: `/v3/users/6de75ba5114649f389a53109323096dd` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:45.762517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860c11d-c4e6-4b29-98ff-b345fa7d433a None admin] RBAC: Authorizing `identity:get_user(user_id=6de75ba5114649f389a53109323096dd)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:45.763942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860c11d-c4e6-4b29-98ff-b345fa7d433a None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:45.765576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 567/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:45 2022] GET /identity/v3/users/6de75ba5114649f389a53109323096dd => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:45.771697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c5254-62f0-4ff8-afa6-6d75857e4aae None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:45.858189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:45.877450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c5254-62f0-4ff8-afa6-6d75857e4aae None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:45.912635 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478055a-cd9f-45ed-aa7b-256a8abd3734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:45.913497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478055a-cd9f-45ed-aa7b-256a8abd3734 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:45.913784 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478055a-cd9f-45ed-aa7b-256a8abd3734 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:45.914055 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478055a-cd9f-45ed-aa7b-256a8abd3734 None admin] PATH_INFO: `/v3/projects/41a7bff642274a40b61c5b161277b623` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:45.915269 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478055a-cd9f-45ed-aa7b-256a8abd3734 None admin] RBAC: Authorizing `identity:get_project(project_id=41a7bff642274a40b61c5b161277b623)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:45.916703 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478055a-cd9f-45ed-aa7b-256a8abd3734 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:45.919195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 585/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:45 2022] GET /identity/v3/projects/41a7bff642274a40b61c5b161277b623 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:45.925803 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860c11d-c4e6-4b29-98ff-b345fa7d433a None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:45.952325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:46.028696 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860c11d-c4e6-4b29-98ff-b345fa7d433a None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:46.062794 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef8f69-cc2d-4e97-a14b-ed9be528dd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:46.063683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef8f69-cc2d-4e97-a14b-ed9be528dd96 None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:46.063966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef8f69-cc2d-4e97-a14b-ed9be528dd96 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:46.064241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef8f69-cc2d-4e97-a14b-ed9be528dd96 None admin] PATH_INFO: `/v3/projects/41a7bff642274a40b61c5b161277b623/users/6de75ba5114649f389a53109323096dd/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:46.066217 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef8f69-cc2d-4e97-a14b-ed9be528dd96 None admin] RBAC: Authorizing `identity:create_grant(project_id=41a7bff642274a40b61c5b161277b623, user_id=6de75ba5114649f389a53109323096dd, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:46.067670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef8f69-cc2d-4e97-a14b-ed9be528dd96 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:46.159184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 568/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:44:45 2022] PUT /identity/v3/projects/41a7bff642274a40b61c5b161277b623/users/6de75ba5114649f389a53109323096dd/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 18:44:48.248172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7e1b4-41d6-4a20-963b-d0bb410437b2 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:48.248778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7e1b4-41d6-4a20-963b-d0bb410437b2 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:48.248778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7e1b4-41d6-4a20-963b-d0bb410437b2 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:48.250160 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 586/1154] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:48.255404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf35373-cc8b-45fe-afe9-74daf59d8f54 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:48.255803 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf35373-cc8b-45fe-afe9-74daf59d8f54 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:48.255948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf35373-cc8b-45fe-afe9-74daf59d8f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:48.275080 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-dbf35373-cc8b-45fe-afe9-74daf59d8f54 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:48.276936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-dbf35373-cc8b-45fe-afe9-74daf59d8f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:48.347638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 569/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:48.372033 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7e1b4-41d6-4a20-963b-d0bb410437b2 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:48.481145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7e1b4-41d6-4a20-963b-d0bb410437b2 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:48.517979 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42136623-699b-4298-9c92-0801a3e6533f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:48.518802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42136623-699b-4298-9c92-0801a3e6533f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:48.519093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42136623-699b-4298-9c92-0801a3e6533f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:48.519371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42136623-699b-4298-9c92-0801a3e6533f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:48.529997 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42136623-699b-4298-9c92-0801a3e6533f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-42136623-699b-4298-9c92-0801a3e6533f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:44:48.537654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:48.538750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:48.539613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:48.539613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:48.539613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:48.539613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:48.539613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:44:48.539613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:44:48.539613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:44:48.539613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:44:48.539613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:44:48.539613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:44:48.539613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:44:48.539613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:44:48.539613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 587/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:44:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:48.545558 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf35373-cc8b-45fe-afe9-74daf59d8f54 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:48.561393 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:48.652786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf35373-cc8b-45fe-afe9-74daf59d8f54 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:48.688760 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066adfb-7f03-4d3b-89f3-5a73f26019a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:48.689551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066adfb-7f03-4d3b-89f3-5a73f26019a3 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:48.689851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066adfb-7f03-4d3b-89f3-5a73f26019a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:48.690130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066adfb-7f03-4d3b-89f3-5a73f26019a3 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:48.690978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066adfb-7f03-4d3b-89f3-5a73f26019a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:48.691293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066adfb-7f03-4d3b-89f3-5a73f26019a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:48.692672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066adfb-7f03-4d3b-89f3-5a73f26019a3 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:48.701368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 570/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:44:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:48.707320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42136623-699b-4298-9c92-0801a3e6533f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:48.721425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:48.813999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42136623-699b-4298-9c92-0801a3e6533f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:48.847995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7cd16-17a1-4998-85bc-bfffa47696cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:48.848745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7cd16-17a1-4998-85bc-bfffa47696cf None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:48.849013 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7cd16-17a1-4998-85bc-bfffa47696cf None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:48.849288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7cd16-17a1-4998-85bc-bfffa47696cf None admin] PATH_INFO: `/v3/users/6de75ba5114649f389a53109323096dd` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:48.850494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7cd16-17a1-4998-85bc-bfffa47696cf None admin] RBAC: Authorizing `identity:get_user(user_id=6de75ba5114649f389a53109323096dd)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:48.851917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7cd16-17a1-4998-85bc-bfffa47696cf None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:48.853553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 588/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:44:48 2022] GET /identity/v3/users/6de75ba5114649f389a53109323096dd => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:48.860211 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066adfb-7f03-4d3b-89f3-5a73f26019a3 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:48.940975 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:48.964756 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066adfb-7f03-4d3b-89f3-5a73f26019a3 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:49.001311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33aeda-772b-4c4b-a59b-5750e0ccb79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:49.001981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33aeda-772b-4c4b-a59b-5750e0ccb79c None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:49.002271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33aeda-772b-4c4b-a59b-5750e0ccb79c None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:49.002603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33aeda-772b-4c4b-a59b-5750e0ccb79c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:49.072413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33aeda-772b-4c4b-a59b-5750e0ccb79c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:49.073990 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33aeda-772b-4c4b-a59b-5750e0ccb79c None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:49.156009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:49.179977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 571/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:49.186661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7cd16-17a1-4998-85bc-bfffa47696cf None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:49.268616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:49.290419 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7cd16-17a1-4998-85bc-bfffa47696cf None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:49.327407 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcab08-735e-453d-9074-7ec9f60f1fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:49.328190 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcab08-735e-453d-9074-7ec9f60f1fae None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:49.328473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcab08-735e-453d-9074-7ec9f60f1fae None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:49.328751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcab08-735e-453d-9074-7ec9f60f1fae None admin] PATH_INFO: `/v3/projects/41a7bff642274a40b61c5b161277b623` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:49.330032 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcab08-735e-453d-9074-7ec9f60f1fae None admin] RBAC: Authorizing `identity:get_project(project_id=41a7bff642274a40b61c5b161277b623)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:49.331579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcab08-735e-453d-9074-7ec9f60f1fae None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:49.333289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 589/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:44:49 2022] GET /identity/v3/projects/41a7bff642274a40b61c5b161277b623 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:49.339581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33aeda-772b-4c4b-a59b-5750e0ccb79c None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:49.440523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33aeda-772b-4c4b-a59b-5750e0ccb79c None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:49.474866 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24748d84-8b4d-40cc-8a9f-d59eeb7c489e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:49.475578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24748d84-8b4d-40cc-8a9f-d59eeb7c489e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:49.475868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24748d84-8b4d-40cc-8a9f-d59eeb7c489e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:49.476150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24748d84-8b4d-40cc-8a9f-d59eeb7c489e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:49.476982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24748d84-8b4d-40cc-8a9f-d59eeb7c489e None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=41a7bff642274a40b61c5b161277b623, user.id=6de75ba5114649f389a53109323096dd) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:49.477292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24748d84-8b4d-40cc-8a9f-d59eeb7c489e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:49.478800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24748d84-8b4d-40cc-8a9f-d59eeb7c489e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:49.486009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 572/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:44:49 2022] GET /identity/v3/role_assignments?user.id=6de75ba5114649f389a53109323096dd&scope.project.id=41a7bff642274a40b61c5b161277b623&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:51.551889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28a233-99c2-4472-b2e6-cdb5267e56d1 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:51.552463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28a233-99c2-4472-b2e6-cdb5267e56d1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:51.552463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28a233-99c2-4472-b2e6-cdb5267e56d1 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:51.553816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 590/1162] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:51.559783 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816b0a-b89c-4adc-ba6a-98e0cc45e095 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:51.560195 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816b0a-b89c-4adc-ba6a-98e0cc45e095 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:51.560344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816b0a-b89c-4adc-ba6a-98e0cc45e095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:51.578711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-5f816b0a-b89c-4adc-ba6a-98e0cc45e095 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:51.579929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5f816b0a-b89c-4adc-ba6a-98e0cc45e095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:51.616354 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:51.651437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 573/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:51.674751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28a233-99c2-4472-b2e6-cdb5267e56d1 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:51.779937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28a233-99c2-4472-b2e6-cdb5267e56d1 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:51.818909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3965a-0a88-4081-9ff3-dafa8edbd62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:51.819485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3965a-0a88-4081-9ff3-dafa8edbd62e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:51.819768 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3965a-0a88-4081-9ff3-dafa8edbd62e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:51.820048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3965a-0a88-4081-9ff3-dafa8edbd62e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:51.835767 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:51.889362 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3965a-0a88-4081-9ff3-dafa8edbd62e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:51.890860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3965a-0a88-4081-9ff3-dafa8edbd62e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:51.991516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 591/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:44:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:44:51.998427 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f816b0a-b89c-4adc-ba6a-98e0cc45e095 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:52.098573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f816b0a-b89c-4adc-ba6a-98e0cc45e095 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:52.133829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca04d8-13ed-469d-8dfb-ff45209b8a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:52.134673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca04d8-13ed-469d-8dfb-ff45209b8a88 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:52.134961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca04d8-13ed-469d-8dfb-ff45209b8a88 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:52.135243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca04d8-13ed-469d-8dfb-ff45209b8a88 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:52.142480 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca04d8-13ed-469d-8dfb-ff45209b8a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:52.143908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca04d8-13ed-469d-8dfb-ff45209b8a88 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-08ca04d8-13ed-469d-8dfb-ff45209b8a88 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:44:52.159364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:44:52.160420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:44:52.161445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 574/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 18:44:51 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:44:52.166727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3965a-0a88-4081-9ff3-dafa8edbd62e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:52.214953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:52.271660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3965a-0a88-4081-9ff3-dafa8edbd62e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:52.310224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41d3de-2232-4f30-82e6-4d9e6e2946e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:52.310938 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41d3de-2232-4f30-82e6-4d9e6e2946e7 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:52.311229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41d3de-2232-4f30-82e6-4d9e6e2946e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:52.311519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41d3de-2232-4f30-82e6-4d9e6e2946e7 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:52.312232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41d3de-2232-4f30-82e6-4d9e6e2946e7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:44:52.312546 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41d3de-2232-4f30-82e6-4d9e6e2946e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:52.313930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41d3de-2232-4f30-82e6-4d9e6e2946e7 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:52.321772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 592/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 18:44:52 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:54.379183 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7218066-f398-4be4-ba28-ffd46c36c564 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:54.379739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7218066-f398-4be4-ba28-ffd46c36c564 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:54.379739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7218066-f398-4be4-ba28-ffd46c36c564 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:54.381162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 575/1167] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:54.386745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093e697-c342-4355-8359-5e8d03cb1da3 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:54.387243 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093e697-c342-4355-8359-5e8d03cb1da3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:54.387420 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093e697-c342-4355-8359-5e8d03cb1da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:54.407326 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d093e697-c342-4355-8359-5e8d03cb1da3 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:54.408574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d093e697-c342-4355-8359-5e8d03cb1da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:54.476800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 593/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:44:54.500785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7218066-f398-4be4-ba28-ffd46c36c564 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:54.533912 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:54.610488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7218066-f398-4be4-ba28-ffd46c36c564 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:54.646755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc84ae-418c-4f9d-94ee-c1669b61a8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:54.647429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc84ae-418c-4f9d-94ee-c1669b61a8c1 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:54.647789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc84ae-418c-4f9d-94ee-c1669b61a8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:54.647987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc84ae-418c-4f9d-94ee-c1669b61a8c1 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:54.648729 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc84ae-418c-4f9d-94ee-c1669b61a8c1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:54.650107 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc84ae-418c-4f9d-94ee-c1669b61a8c1 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:54.753789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 576/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Mon Jul 25 18:44:54 2022] POST /identity/v3/domains => generated 267 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:44:56.808479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55584a2-9e4b-43c9-91d5-66331ef5b392 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:56.809049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55584a2-9e4b-43c9-91d5-66331ef5b392 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:56.809153 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55584a2-9e4b-43c9-91d5-66331ef5b392 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:56.810607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 594/1170] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:56.815957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326167e-3417-48c1-be9c-29e2d596a643 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:56.816413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326167e-3417-48c1-be9c-29e2d596a643 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:56.816506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326167e-3417-48c1-be9c-29e2d596a643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:56.835021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-5326167e-3417-48c1-be9c-29e2d596a643 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:56.836235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5326167e-3417-48c1-be9c-29e2d596a643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:56.905948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 577/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:56.932158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55584a2-9e4b-43c9-91d5-66331ef5b392 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:56.977982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:57.039997 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55584a2-9e4b-43c9-91d5-66331ef5b392 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:57.076359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55eac2-4a9f-4430-b4ca-4bb24f2f6feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:57.077144 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55eac2-4a9f-4430-b4ca-4bb24f2f6feb None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:57.077429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55eac2-4a9f-4430-b4ca-4bb24f2f6feb None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:57.077707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55eac2-4a9f-4430-b4ca-4bb24f2f6feb None admin] PATH_INFO: `/v3/domains/7d0ffe25b790452e96d432c0ccc0f05b` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:57.089856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55eac2-4a9f-4430-b4ca-4bb24f2f6feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=7d0ffe25b790452e96d432c0ccc0f05b)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:57.091544 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55eac2-4a9f-4430-b4ca-4bb24f2f6feb None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:57.093291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 595/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Mon Jul 25 18:44:56 2022] GET /identity/v3/domains/7d0ffe25b790452e96d432c0ccc0f05b => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:57.099415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326167e-3417-48c1-be9c-29e2d596a643 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:57.122547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:57.203493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326167e-3417-48c1-be9c-29e2d596a643 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:57.238066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660902d9-10da-4fbe-8329-edf2bd310fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:57.238767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660902d9-10da-4fbe-8329-edf2bd310fac None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:57.239148 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660902d9-10da-4fbe-8329-edf2bd310fac None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:57.239436 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660902d9-10da-4fbe-8329-edf2bd310fac None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:57.240166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660902d9-10da-4fbe-8329-edf2bd310fac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:57.241613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660902d9-10da-4fbe-8329-edf2bd310fac None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:57.427480 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 578/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:44:57 2022] POST /identity/v3/projects => generated 361 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:44:59.501499 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b124856-704a-4092-9adc-7e81a9e07ae2 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:59.502026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b124856-704a-4092-9adc-7e81a9e07ae2 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:59.502026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b124856-704a-4092-9adc-7e81a9e07ae2 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:59.503475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 596/1174] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:44:59.509229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50307acf-aa14-45bb-8853-de7ec9014af7 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:59.509689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50307acf-aa14-45bb-8853-de7ec9014af7 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:59.509783 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50307acf-aa14-45bb-8853-de7ec9014af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:59.529759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-50307acf-aa14-45bb-8853-de7ec9014af7 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:44:59.531015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-50307acf-aa14-45bb-8853-de7ec9014af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:44:59.601840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 579/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:44:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:44:59.627646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b124856-704a-4092-9adc-7e81a9e07ae2 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:59.728259 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:59.732874 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b124856-704a-4092-9adc-7e81a9e07ae2 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:59.769251 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95275624-5cee-4a89-be6e-24407ef5b1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:59.770261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95275624-5cee-4a89-be6e-24407ef5b1aa None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:59.770772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95275624-5cee-4a89-be6e-24407ef5b1aa None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:59.771557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95275624-5cee-4a89-be6e-24407ef5b1aa None admin] PATH_INFO: `/v3/domains/7d0ffe25b790452e96d432c0ccc0f05b` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:59.772824 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95275624-5cee-4a89-be6e-24407ef5b1aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=7d0ffe25b790452e96d432c0ccc0f05b)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:59.774296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95275624-5cee-4a89-be6e-24407ef5b1aa None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:44:59.776090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 597/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Mon Jul 25 18:44:59 2022] GET /identity/v3/domains/7d0ffe25b790452e96d432c0ccc0f05b => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:44:59.782024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50307acf-aa14-45bb-8853-de7ec9014af7 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:44:59.784285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:44:59.892029 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50307acf-aa14-45bb-8853-de7ec9014af7 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:44:59.929146 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f956ac0-8ebd-403c-8403-4366733e847d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:44:59.929826 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f956ac0-8ebd-403c-8403-4366733e847d None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:44:59.930114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f956ac0-8ebd-403c-8403-4366733e847d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:44:59.930424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f956ac0-8ebd-403c-8403-4366733e847d None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:44:59.931156 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f956ac0-8ebd-403c-8403-4366733e847d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:44:59.932601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f956ac0-8ebd-403c-8403-4366733e847d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:00.051592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 580/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:44:59 2022] POST /identity/v3/users => generated 311 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:45:02.176523 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e52ace-4ca2-4d0e-aef4-a66fd81b8147 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:02.177027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e52ace-4ca2-4d0e-aef4-a66fd81b8147 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:02.177121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e52ace-4ca2-4d0e-aef4-a66fd81b8147 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:02.178494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 598/1178] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:45:02.183905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4db53a-ef0c-4c8c-bf68-9a902afe731b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:02.184349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4db53a-ef0c-4c8c-bf68-9a902afe731b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:02.184467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4db53a-ef0c-4c8c-bf68-9a902afe731b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:02.203287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3f4db53a-ef0c-4c8c-bf68-9a902afe731b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:45:02.204477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3f4db53a-ef0c-4c8c-bf68-9a902afe731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:45:02.258821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:02.272869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 581/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:45:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:45:02.297628 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e52ace-4ca2-4d0e-aef4-a66fd81b8147 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:02.404301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e52ace-4ca2-4d0e-aef4-a66fd81b8147 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:02.439528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd318f08-81e5-4e34-8e6a-f857c7ab6db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:02.440354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd318f08-81e5-4e34-8e6a-f857c7ab6db4 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:02.440645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd318f08-81e5-4e34-8e6a-f857c7ab6db4 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:02.440930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd318f08-81e5-4e34-8e6a-f857c7ab6db4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:02.452217 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd318f08-81e5-4e34-8e6a-f857c7ab6db4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-fd318f08-81e5-4e34-8e6a-f857c7ab6db4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:45:02.460092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:45:02.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:45:02.462074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:45:02.462074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:45:02.462074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:45:02.462074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:45:02.462074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:45:02.462074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:45:02.462074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:45:02.462074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:45:02.462074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:45:02.462074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:45:02.462074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:02.462074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:45:02.462074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 599/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:45:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:45:02.468068 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4db53a-ef0c-4c8c-bf68-9a902afe731b None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:02.575429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4db53a-ef0c-4c8c-bf68-9a902afe731b None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:02.612547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806587ae-9d1f-436c-a99a-3f5b45bdf3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:02.613224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806587ae-9d1f-436c-a99a-3f5b45bdf3fb None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:02.613509 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806587ae-9d1f-436c-a99a-3f5b45bdf3fb None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:02.613789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806587ae-9d1f-436c-a99a-3f5b45bdf3fb None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:02.614575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806587ae-9d1f-436c-a99a-3f5b45bdf3fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:45:02.614890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806587ae-9d1f-436c-a99a-3f5b45bdf3fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:02.616264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806587ae-9d1f-436c-a99a-3f5b45bdf3fb None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:02.625439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 582/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:45:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:02.632101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd318f08-81e5-4e34-8e6a-f857c7ab6db4 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:02.662928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:02.739410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd318f08-81e5-4e34-8e6a-f857c7ab6db4 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:02.773976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6485944-ee73-48e5-9e3b-983a81c51496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:02.774888 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6485944-ee73-48e5-9e3b-983a81c51496 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:02.775173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6485944-ee73-48e5-9e3b-983a81c51496 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:02.775446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6485944-ee73-48e5-9e3b-983a81c51496 None admin] PATH_INFO: `/v3/users/e0289986fdcc4e30908a703375a0e6eb` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:02.790699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6485944-ee73-48e5-9e3b-983a81c51496 None admin] RBAC: Authorizing `identity:get_user(user_id=e0289986fdcc4e30908a703375a0e6eb)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:02.792205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6485944-ee73-48e5-9e3b-983a81c51496 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:02.794936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 600/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:45:02 2022] GET /identity/v3/users/e0289986fdcc4e30908a703375a0e6eb => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:02.800848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806587ae-9d1f-436c-a99a-3f5b45bdf3fb None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:02.853204 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:02.906081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806587ae-9d1f-436c-a99a-3f5b45bdf3fb None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:02.939957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef972bbe-8499-4c84-8202-cc180d17baab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:02.940605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef972bbe-8499-4c84-8202-cc180d17baab None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:02.940888 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef972bbe-8499-4c84-8202-cc180d17baab None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:02.941166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef972bbe-8499-4c84-8202-cc180d17baab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:03.023074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef972bbe-8499-4c84-8202-cc180d17baab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:03.024437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef972bbe-8499-4c84-8202-cc180d17baab None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:03.072690 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:03.125965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 583/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:45:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:45:03.131998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6485944-ee73-48e5-9e3b-983a81c51496 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:03.220038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:03.236004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6485944-ee73-48e5-9e3b-983a81c51496 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:03.271573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3e3f3-144d-4e62-9f64-bb6bebe1fb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:03.272446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3e3f3-144d-4e62-9f64-bb6bebe1fb90 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:03.272732 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3e3f3-144d-4e62-9f64-bb6bebe1fb90 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:03.273012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3e3f3-144d-4e62-9f64-bb6bebe1fb90 None admin] PATH_INFO: `/v3/projects/c05b0c8cd2a84067bddec96620315df3` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:03.274254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3e3f3-144d-4e62-9f64-bb6bebe1fb90 None admin] RBAC: Authorizing `identity:get_project(project_id=c05b0c8cd2a84067bddec96620315df3)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:03.275656 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3e3f3-144d-4e62-9f64-bb6bebe1fb90 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:03.278093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 601/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:45:03 2022] GET /identity/v3/projects/c05b0c8cd2a84067bddec96620315df3 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:03.284438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef972bbe-8499-4c84-8202-cc180d17baab None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:03.386537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef972bbe-8499-4c84-8202-cc180d17baab None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:03.422776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb22eb9-b05d-4832-8d42-92e0e6499b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:03.423441 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb22eb9-b05d-4832-8d42-92e0e6499b5b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:03.423733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb22eb9-b05d-4832-8d42-92e0e6499b5b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:03.424007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb22eb9-b05d-4832-8d42-92e0e6499b5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:03.424845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb22eb9-b05d-4832-8d42-92e0e6499b5b None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=c05b0c8cd2a84067bddec96620315df3, user.id=e0289986fdcc4e30908a703375a0e6eb) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:45:03.425159 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb22eb9-b05d-4832-8d42-92e0e6499b5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:03.426645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb22eb9-b05d-4832-8d42-92e0e6499b5b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:03.432319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 584/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:45:03 2022] GET /identity/v3/role_assignments?user.id=e0289986fdcc4e30908a703375a0e6eb&scope.project.id=c05b0c8cd2a84067bddec96620315df3&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:05.463748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef40ceb-3c14-484d-8dc6-0be1cc9b6baf None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:05.464382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef40ceb-3c14-484d-8dc6-0be1cc9b6baf None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:05.464382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef40ceb-3c14-484d-8dc6-0be1cc9b6baf None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:05.465699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 602/1186] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:45:05.471257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006ec8e-a24f-42ae-8f0d-737d33cf2ee2 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:05.471714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006ec8e-a24f-42ae-8f0d-737d33cf2ee2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:05.471813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006ec8e-a24f-42ae-8f0d-737d33cf2ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:05.491011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-7006ec8e-a24f-42ae-8f0d-737d33cf2ee2 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:45:05.492228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-7006ec8e-a24f-42ae-8f0d-737d33cf2ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:45:05.502824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:05.560580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 585/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:45:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:45:05.583035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef40ceb-3c14-484d-8dc6-0be1cc9b6baf None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:05.685925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef40ceb-3c14-484d-8dc6-0be1cc9b6baf None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:05.719610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d856d1-ceef-45c0-bda6-a9fe028e4563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:05.720330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d856d1-ceef-45c0-bda6-a9fe028e4563 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:05.720616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d856d1-ceef-45c0-bda6-a9fe028e4563 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:05.720901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d856d1-ceef-45c0-bda6-a9fe028e4563 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:05.731884 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d856d1-ceef-45c0-bda6-a9fe028e4563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-74d856d1-ceef-45c0-bda6-a9fe028e4563 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:45:05.739466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:45:05.740627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:45:05.741883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:45:05.741883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:45:05.741883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:45:05.741883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:45:05.741883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:45:05.741883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:45:05.741883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:45:05.741883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:45:05.741883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:45:05.741883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:45:05.741883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:05.741883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:45:05.741883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 603/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:45:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:45:05.747209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006ec8e-a24f-42ae-8f0d-737d33cf2ee2 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:05.847095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006ec8e-a24f-42ae-8f0d-737d33cf2ee2 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:05.882158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc12911-f547-474e-bbb2-b2ffa8b37232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:05.882847 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc12911-f547-474e-bbb2-b2ffa8b37232 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:05.883137 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc12911-f547-474e-bbb2-b2ffa8b37232 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:05.883420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc12911-f547-474e-bbb2-b2ffa8b37232 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:05.884166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc12911-f547-474e-bbb2-b2ffa8b37232 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:45:05.884477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc12911-f547-474e-bbb2-b2ffa8b37232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:05.885845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc12911-f547-474e-bbb2-b2ffa8b37232 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:05.894802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 586/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:45:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:05.900719 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d856d1-ceef-45c0-bda6-a9fe028e4563 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:05.917948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:06.019254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d856d1-ceef-45c0-bda6-a9fe028e4563 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:06.053207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be82d82-3540-45bd-ac0b-a047216dd6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:06.054085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be82d82-3540-45bd-ac0b-a047216dd6c6 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:06.054392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be82d82-3540-45bd-ac0b-a047216dd6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:06.054678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be82d82-3540-45bd-ac0b-a047216dd6c6 None admin] PATH_INFO: `/v3/users/e0289986fdcc4e30908a703375a0e6eb` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:06.055879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be82d82-3540-45bd-ac0b-a047216dd6c6 None admin] RBAC: Authorizing `identity:get_user(user_id=e0289986fdcc4e30908a703375a0e6eb)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:06.057318 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be82d82-3540-45bd-ac0b-a047216dd6c6 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:06.059022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 604/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:45:05 2022] GET /identity/v3/users/e0289986fdcc4e30908a703375a0e6eb => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:06.066117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc12911-f547-474e-bbb2-b2ffa8b37232 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:06.076864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:06.168215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc12911-f547-474e-bbb2-b2ffa8b37232 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:06.202676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d0ffe-3c47-4d4c-afd3-36b67d6c9a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:06.203535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d0ffe-3c47-4d4c-afd3-36b67d6c9a53 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:06.203825 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d0ffe-3c47-4d4c-afd3-36b67d6c9a53 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:06.204107 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d0ffe-3c47-4d4c-afd3-36b67d6c9a53 None admin] PATH_INFO: `/v3/projects/c05b0c8cd2a84067bddec96620315df3` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:06.205307 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d0ffe-3c47-4d4c-afd3-36b67d6c9a53 None admin] RBAC: Authorizing `identity:get_project(project_id=c05b0c8cd2a84067bddec96620315df3)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:06.206862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d0ffe-3c47-4d4c-afd3-36b67d6c9a53 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:06.208579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 587/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:45:06 2022] GET /identity/v3/projects/c05b0c8cd2a84067bddec96620315df3 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:06.215403 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be82d82-3540-45bd-ac0b-a047216dd6c6 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:06.297930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:06.315958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be82d82-3540-45bd-ac0b-a047216dd6c6 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:06.350723 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b19d8-b9fa-49ba-ad6f-38dca951e55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:06.351619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b19d8-b9fa-49ba-ad6f-38dca951e55f None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:06.351901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b19d8-b9fa-49ba-ad6f-38dca951e55f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:06.352175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b19d8-b9fa-49ba-ad6f-38dca951e55f None admin] PATH_INFO: `/v3/projects/c05b0c8cd2a84067bddec96620315df3/users/e0289986fdcc4e30908a703375a0e6eb/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:06.354149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b19d8-b9fa-49ba-ad6f-38dca951e55f None admin] RBAC: Authorizing `identity:create_grant(project_id=c05b0c8cd2a84067bddec96620315df3, user_id=e0289986fdcc4e30908a703375a0e6eb, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:06.355950 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b19d8-b9fa-49ba-ad6f-38dca951e55f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:06.432475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 605/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:45:06 2022] PUT /identity/v3/projects/c05b0c8cd2a84067bddec96620315df3/users/e0289986fdcc4e30908a703375a0e6eb/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:45:08.433534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de8c45-4719-4e2c-847e-873f37af6370 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:08.434171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de8c45-4719-4e2c-847e-873f37af6370 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:08.434171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de8c45-4719-4e2c-847e-873f37af6370 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:08.435602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 588/1193] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:45:08.440940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c561df9-9d98-483e-b594-877ea67e759a None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:08.441381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c561df9-9d98-483e-b594-877ea67e759a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:08.441493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c561df9-9d98-483e-b594-877ea67e759a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:08.460427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-2c561df9-9d98-483e-b594-877ea67e759a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:45:08.461637 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-2c561df9-9d98-483e-b594-877ea67e759a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:45:08.531909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 606/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:45:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:45:08.556319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de8c45-4719-4e2c-847e-873f37af6370 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:08.637444 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:08.662844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de8c45-4719-4e2c-847e-873f37af6370 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:08.697356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2ab80-4d71-4fb5-a05c-d92da3485d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:08.698069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2ab80-4d71-4fb5-a05c-d92da3485d43 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:08.698380 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2ab80-4d71-4fb5-a05c-d92da3485d43 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:08.698747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2ab80-4d71-4fb5-a05c-d92da3485d43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:08.709611 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2ab80-4d71-4fb5-a05c-d92da3485d43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-50c2ab80-4d71-4fb5-a05c-d92da3485d43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:45:08.717286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:45:08.718474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:45:08.719508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:45:08.719508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:45:08.719508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:45:08.719508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:45:08.719508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:45:08.719508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:45:08.719508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:45:08.719508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:45:08.719508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:45:08.719508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:45:08.719508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:45:08.719508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:45:08.719508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 589/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:45:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:45:08.724754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c561df9-9d98-483e-b594-877ea67e759a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:08.797051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:08.827531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c561df9-9d98-483e-b594-877ea67e759a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:08.862848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defef7cf-22fa-4161-9bf5-d6f907c23a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:08.863906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defef7cf-22fa-4161-9bf5-d6f907c23a83 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:08.864193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defef7cf-22fa-4161-9bf5-d6f907c23a83 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:08.864476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defef7cf-22fa-4161-9bf5-d6f907c23a83 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:08.865281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defef7cf-22fa-4161-9bf5-d6f907c23a83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:45:08.865542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defef7cf-22fa-4161-9bf5-d6f907c23a83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:08.866948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defef7cf-22fa-4161-9bf5-d6f907c23a83 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:08.875802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 607/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:45:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:08.881921 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2ab80-4d71-4fb5-a05c-d92da3485d43 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:08.983343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2ab80-4d71-4fb5-a05c-d92da3485d43 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:09.016821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db3a71-6977-47ac-ae16-dcb3884d7ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:09.017690 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db3a71-6977-47ac-ae16-dcb3884d7ea6 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:09.017974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db3a71-6977-47ac-ae16-dcb3884d7ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:09.018248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db3a71-6977-47ac-ae16-dcb3884d7ea6 None admin] PATH_INFO: `/v3/users/e0289986fdcc4e30908a703375a0e6eb` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:09.019461 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db3a71-6977-47ac-ae16-dcb3884d7ea6 None admin] RBAC: Authorizing `identity:get_user(user_id=e0289986fdcc4e30908a703375a0e6eb)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:09.020865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db3a71-6977-47ac-ae16-dcb3884d7ea6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:09.022529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 590/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 18:45:08 2022] GET /identity/v3/users/e0289986fdcc4e30908a703375a0e6eb => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:09.028751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defef7cf-22fa-4161-9bf5-d6f907c23a83 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:09.134730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defef7cf-22fa-4161-9bf5-d6f907c23a83 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:09.169600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9a3e2-d316-4803-ab4c-03a571a7361d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:09.170266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9a3e2-d316-4803-ab4c-03a571a7361d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:09.170588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9a3e2-d316-4803-ab4c-03a571a7361d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:09.170871 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9a3e2-d316-4803-ab4c-03a571a7361d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:09.173643 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:09.240955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9a3e2-d316-4803-ab4c-03a571a7361d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:09.242313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9a3e2-d316-4803-ab4c-03a571a7361d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:09.342862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 608/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:45:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:45:09.349026 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db3a71-6977-47ac-ae16-dcb3884d7ea6 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:09.352036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:09.454117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db3a71-6977-47ac-ae16-dcb3884d7ea6 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:09.489681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30b7ff-a523-46be-9354-6e72f9ee6bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:09.490589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30b7ff-a523-46be-9354-6e72f9ee6bb0 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:09.490883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30b7ff-a523-46be-9354-6e72f9ee6bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:09.491219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30b7ff-a523-46be-9354-6e72f9ee6bb0 None admin] PATH_INFO: `/v3/projects/c05b0c8cd2a84067bddec96620315df3` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:09.492357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30b7ff-a523-46be-9354-6e72f9ee6bb0 None admin] RBAC: Authorizing `identity:get_project(project_id=c05b0c8cd2a84067bddec96620315df3)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:09.493834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30b7ff-a523-46be-9354-6e72f9ee6bb0 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:09.495598 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 591/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 18:45:09 2022] GET /identity/v3/projects/c05b0c8cd2a84067bddec96620315df3 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:09.502296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9a3e2-d316-4803-ab4c-03a571a7361d None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:45:09.535385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:09.602001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9a3e2-d316-4803-ab4c-03a571a7361d None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:45:09.638408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d214fd0-89ef-44bd-b1f5-bf8694e477c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:45:09.639065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d214fd0-89ef-44bd-b1f5-bf8694e477c3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:45:09.639352 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d214fd0-89ef-44bd-b1f5-bf8694e477c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:45:09.639727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d214fd0-89ef-44bd-b1f5-bf8694e477c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:45:09.640469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d214fd0-89ef-44bd-b1f5-bf8694e477c3 None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=c05b0c8cd2a84067bddec96620315df3, user.id=e0289986fdcc4e30908a703375a0e6eb) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:45:09.640784 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d214fd0-89ef-44bd-b1f5-bf8694e477c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:45:09.642222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d214fd0-89ef-44bd-b1f5-bf8694e477c3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:45:09.648156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 609/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:45:09 2022] GET /identity/v3/role_assignments?user.id=e0289986fdcc4e30908a703375a0e6eb&scope.project.id=c05b0c8cd2a84067bddec96620315df3&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:45:19.485376 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=65 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:45:19.645439 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=51 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:34.993519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2432d0-412a-458d-87fb-f8a63786b2cd None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:34.994122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2432d0-412a-458d-87fb-f8a63786b2cd None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:34.994122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2432d0-412a-458d-87fb-f8a63786b2cd None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:34.995542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 592/1201] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:46:35.001817 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33606a3-5280-4803-9c32-3122e6e393dc None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:35.002371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33606a3-5280-4803-9c32-3122e6e393dc None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:35.002498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33606a3-5280-4803-9c32-3122e6e393dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:35.022006 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d33606a3-5280-4803-9c32-3122e6e393dc None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:35.023849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d33606a3-5280-4803-9c32-3122e6e393dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:35.122155 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 610/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 18:46:35.149879 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2432d0-412a-458d-87fb-f8a63786b2cd None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:35.281280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2432d0-412a-458d-87fb-f8a63786b2cd None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:35.325059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced522c6-f267-4f87-84b5-15e06af8420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:35.325875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced522c6-f267-4f87-84b5-15e06af8420c None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:35.326156 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced522c6-f267-4f87-84b5-15e06af8420c None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:35.326460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced522c6-f267-4f87-84b5-15e06af8420c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:35.339724 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced522c6-f267-4f87-84b5-15e06af8420c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:35.341197 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced522c6-f267-4f87-84b5-15e06af8420c None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-ced522c6-f267-4f87-84b5-15e06af8420c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:35.360619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:35.367283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:35.368277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:35.368277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:35.368277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:35.368277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:35.368277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 593/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:35.368646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33606a3-5280-4803-9c32-3122e6e393dc None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:35.480391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33606a3-5280-4803-9c32-3122e6e393dc None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:35.512453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:35.518580 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96023c-2d3b-4947-8c73-86374eebbb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:35.519266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96023c-2d3b-4947-8c73-86374eebbb92 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:35.519549 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96023c-2d3b-4947-8c73-86374eebbb92 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:35.519824 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96023c-2d3b-4947-8c73-86374eebbb92 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:35.521292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96023c-2d3b-4947-8c73-86374eebbb92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:35.521786 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96023c-2d3b-4947-8c73-86374eebbb92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:35.523200 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96023c-2d3b-4947-8c73-86374eebbb92 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:35.536127 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 611/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:35.542808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced522c6-f267-4f87-84b5-15e06af8420c None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:35.600062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:35.653752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced522c6-f267-4f87-84b5-15e06af8420c None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:35.689674 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a31ec-475c-445f-aa8e-7f62a949d6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:35.692618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a31ec-475c-445f-aa8e-7f62a949d6e3 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:35.692618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a31ec-475c-445f-aa8e-7f62a949d6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:35.692618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a31ec-475c-445f-aa8e-7f62a949d6e3 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:35.692618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a31ec-475c-445f-aa8e-7f62a949d6e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:35.694047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a31ec-475c-445f-aa8e-7f62a949d6e3 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:35.803367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 594/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 18:46:35 2022] POST /identity/v3/users => generated 251 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:46:37.976078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea723491-7094-4bbb-8719-a3d851fd87ce None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:37.976711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea723491-7094-4bbb-8719-a3d851fd87ce None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:37.976711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea723491-7094-4bbb-8719-a3d851fd87ce None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:37.978148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 612/1206] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:46:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:46:37.983770 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7309669-a219-4200-a370-ab1bd41110ec None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:37.984249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7309669-a219-4200-a370-ab1bd41110ec None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:37.984336 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7309669-a219-4200-a370-ab1bd41110ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:38.002808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e7309669-a219-4200-a370-ab1bd41110ec None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:38.014026 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e7309669-a219-4200-a370-ab1bd41110ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:38.075541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 595/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:46:38.100794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea723491-7094-4bbb-8719-a3d851fd87ce None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:38.211560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea723491-7094-4bbb-8719-a3d851fd87ce None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:38.250131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a4579-c75f-4c14-8467-a4622354cc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:38.250893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a4579-c75f-4c14-8467-a4622354cc0a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:38.251195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a4579-c75f-4c14-8467-a4622354cc0a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:38.251463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a4579-c75f-4c14-8467-a4622354cc0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:38.262955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a4579-c75f-4c14-8467-a4622354cc0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-9f8a4579-c75f-4c14-8467-a4622354cc0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:46:38.271466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:38.274399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:38.275491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:38.275491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:38.275491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:38.275491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:38.275491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:46:38.275491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:46:38.275491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:46:38.275491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:46:38.275491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:46:38.275491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:46:38.275491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:46:38.275491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:46:38.275491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 613/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:46:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:38.279234 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7309669-a219-4200-a370-ab1bd41110ec None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:38.322795 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:38.386344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7309669-a219-4200-a370-ab1bd41110ec None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:38.421205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60695570-faf5-49ee-9e25-2ec1ac31e8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:38.421876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60695570-faf5-49ee-9e25-2ec1ac31e8ff None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:38.422162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60695570-faf5-49ee-9e25-2ec1ac31e8ff None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:38.422564 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60695570-faf5-49ee-9e25-2ec1ac31e8ff None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:38.423233 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60695570-faf5-49ee-9e25-2ec1ac31e8ff None admin] RBAC: Adding query filter params (name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:38.423548 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60695570-faf5-49ee-9e25-2ec1ac31e8ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:38.424916 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60695570-faf5-49ee-9e25-2ec1ac31e8ff None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:38.434091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 596/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:46:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:38.439881 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a4579-c75f-4c14-8467-a4622354cc0a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:38.492274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:38.546424 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a4579-c75f-4c14-8467-a4622354cc0a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:38.583406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661504f9-d71c-4486-82b4-25db78a55696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:38.584202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661504f9-d71c-4486-82b4-25db78a55696 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:38.584488 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661504f9-d71c-4486-82b4-25db78a55696 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:38.584766 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661504f9-d71c-4486-82b4-25db78a55696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:38.595678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661504f9-d71c-4486-82b4-25db78a55696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:38.597205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661504f9-d71c-4486-82b4-25db78a55696 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-661504f9-d71c-4486-82b4-25db78a55696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:38.616473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:38.617756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:38.618710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:38.618710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:38.618710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:38.618710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:46:38.618710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 614/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:38.623616 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60695570-faf5-49ee-9e25-2ec1ac31e8ff None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:38.731420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:38.734436 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60695570-faf5-49ee-9e25-2ec1ac31e8ff None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:38.773432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b15b3-7a4e-4f72-81d0-027d7524f66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:38.774141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b15b3-7a4e-4f72-81d0-027d7524f66b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:38.774460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b15b3-7a4e-4f72-81d0-027d7524f66b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:38.774747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b15b3-7a4e-4f72-81d0-027d7524f66b None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:38.775493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b15b3-7a4e-4f72-81d0-027d7524f66b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:38.775807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b15b3-7a4e-4f72-81d0-027d7524f66b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:38.777203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b15b3-7a4e-4f72-81d0-027d7524f66b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:38.790298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 597/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:38.796220 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661504f9-d71c-4486-82b4-25db78a55696 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:38.910153 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661504f9-d71c-4486-82b4-25db78a55696 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:38.911958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:38.949922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b896faf-0e54-4e9a-930d-2029e9dde341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:38.950773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b896faf-0e54-4e9a-930d-2029e9dde341 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:38.951064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b896faf-0e54-4e9a-930d-2029e9dde341 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:38.951345 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b896faf-0e54-4e9a-930d-2029e9dde341 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:38.959376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b896faf-0e54-4e9a-930d-2029e9dde341 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:38.960826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b896faf-0e54-4e9a-930d-2029e9dde341 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-3b896faf-0e54-4e9a-930d-2029e9dde341 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:46:38.975907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:38.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:38.984748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:38.984748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:38.984748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:46:38.984748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:46:38.984748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:46:38.984748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:46:38.984748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:46:38.984748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:46:38.984748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:38.984748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:46:38.984748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 615/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:38.985926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b15b3-7a4e-4f72-81d0-027d7524f66b None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:39.100535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b15b3-7a4e-4f72-81d0-027d7524f66b None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:39.142587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946539d-2c0d-4baf-bb8e-ab3bff09fa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:39.143285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946539d-2c0d-4baf-bb8e-ab3bff09fa6e None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:39.143579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946539d-2c0d-4baf-bb8e-ab3bff09fa6e None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:39.143900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946539d-2c0d-4baf-bb8e-ab3bff09fa6e None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:39.144752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946539d-2c0d-4baf-bb8e-ab3bff09fa6e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:39.145073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946539d-2c0d-4baf-bb8e-ab3bff09fa6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:39.146595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946539d-2c0d-4baf-bb8e-ab3bff09fa6e None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:39.166056 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 598/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Jul 25 18:46:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:39.171893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b896faf-0e54-4e9a-930d-2029e9dde341 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:39.280240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b896faf-0e54-4e9a-930d-2029e9dde341 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:39.318047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71668cce-bcd4-4084-939d-b3cc88c1069f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:39.324081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71668cce-bcd4-4084-939d-b3cc88c1069f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:39.324081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71668cce-bcd4-4084-939d-b3cc88c1069f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:39.324081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71668cce-bcd4-4084-939d-b3cc88c1069f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:39.350083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:39.389732 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71668cce-bcd4-4084-939d-b3cc88c1069f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:39.391262 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71668cce-bcd4-4084-939d-b3cc88c1069f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:39.496619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 616/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:46:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:46:39.502665 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946539d-2c0d-4baf-bb8e-ab3bff09fa6e None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:39.523997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:39.635351 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946539d-2c0d-4baf-bb8e-ab3bff09fa6e None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:39.675940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef66a84-bcfd-4ad7-bf05-f08933b76406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:39.676799 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef66a84-bcfd-4ad7-bf05-f08933b76406 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:39.677080 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef66a84-bcfd-4ad7-bf05-f08933b76406 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:39.677355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef66a84-bcfd-4ad7-bf05-f08933b76406 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:39.688606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef66a84-bcfd-4ad7-bf05-f08933b76406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:39.690073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef66a84-bcfd-4ad7-bf05-f08933b76406 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-9ef66a84-bcfd-4ad7-bf05-f08933b76406 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:39.708966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:39.726378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:39.727489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:39.727489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:39.727489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:39.727489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:39.727489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 599/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:39.727941 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71668cce-bcd4-4084-939d-b3cc88c1069f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:39.794803 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:39.836089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71668cce-bcd4-4084-939d-b3cc88c1069f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:39.871996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b09ff-e8c2-42bf-ab0f-c941862899aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:39.872726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b09ff-e8c2-42bf-ab0f-c941862899aa None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:39.873113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b09ff-e8c2-42bf-ab0f-c941862899aa None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:39.873423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b09ff-e8c2-42bf-ab0f-c941862899aa None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:39.874210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b09ff-e8c2-42bf-ab0f-c941862899aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:39.874585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b09ff-e8c2-42bf-ab0f-c941862899aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:39.876231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b09ff-e8c2-42bf-ab0f-c941862899aa None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:39.890070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 617/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:39.896742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef66a84-bcfd-4ad7-bf05-f08933b76406 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:39.974813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:40.011046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef66a84-bcfd-4ad7-bf05-f08933b76406 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:40.049251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74bd9a-7cb3-4acb-9eb2-0a59ffdc1c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:40.049992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74bd9a-7cb3-4acb-9eb2-0a59ffdc1c0b None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:40.050284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74bd9a-7cb3-4acb-9eb2-0a59ffdc1c0b None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:40.050603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74bd9a-7cb3-4acb-9eb2-0a59ffdc1c0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:40.058061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74bd9a-7cb3-4acb-9eb2-0a59ffdc1c0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:40.059557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74bd9a-7cb3-4acb-9eb2-0a59ffdc1c0b None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-6e74bd9a-7cb3-4acb-9eb2-0a59ffdc1c0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:46:40.074523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:40.075640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:40.076741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:46:40.076741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:46:40.076741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:46:40.076741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:46:40.076741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:46:40.076741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:46:40.076741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:40.076741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:40.076741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 600/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:46:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:40.082573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b09ff-e8c2-42bf-ab0f-c941862899aa None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:40.200376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b09ff-e8c2-42bf-ab0f-c941862899aa None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:40.235600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:40.252158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213483b0-7482-40c9-9ee0-51776c6c523c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:40.253052 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213483b0-7482-40c9-9ee0-51776c6c523c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:40.253383 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213483b0-7482-40c9-9ee0-51776c6c523c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:40.253705 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213483b0-7482-40c9-9ee0-51776c6c523c None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:40.254667 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213483b0-7482-40c9-9ee0-51776c6c523c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:40.255026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213483b0-7482-40c9-9ee0-51776c6c523c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:40.256575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213483b0-7482-40c9-9ee0-51776c6c523c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:40.271636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 618/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:46:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:40.278581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74bd9a-7cb3-4acb-9eb2-0a59ffdc1c0b None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:40.392719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74bd9a-7cb3-4acb-9eb2-0a59ffdc1c0b None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:40.425729 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:40.432274 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf44f36-d777-4b59-a87e-9cb9f3fb355d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:40.440537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf44f36-d777-4b59-a87e-9cb9f3fb355d None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:40.440537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf44f36-d777-4b59-a87e-9cb9f3fb355d None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:40.440537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf44f36-d777-4b59-a87e-9cb9f3fb355d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:40.440537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf44f36-d777-4b59-a87e-9cb9f3fb355d None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=f5027a60da214afda9144b5ae6cce15b) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:40.440537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf44f36-d777-4b59-a87e-9cb9f3fb355d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:40.440537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf44f36-d777-4b59-a87e-9cb9f3fb355d None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:40.442698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 601/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:46:40 2022] GET /identity/v3/role_assignments?user.id=f5027a60da214afda9144b5ae6cce15b&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:42.493365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb3e0e-47e2-40f6-806f-fc453376f81e None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:42.498346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb3e0e-47e2-40f6-806f-fc453376f81e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:42.498346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb3e0e-47e2-40f6-806f-fc453376f81e None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:42.498346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 619/1220] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:46:42.500873 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f1edf-4f1a-4637-9c57-8045bf95a166 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:42.501329 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f1edf-4f1a-4637-9c57-8045bf95a166 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:42.501474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f1edf-4f1a-4637-9c57-8045bf95a166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:42.521707 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-001f1edf-4f1a-4637-9c57-8045bf95a166 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:42.522950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-001f1edf-4f1a-4637-9c57-8045bf95a166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:42.600545 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 602/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:46:42.623295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb3e0e-47e2-40f6-806f-fc453376f81e None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:42.740693 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb3e0e-47e2-40f6-806f-fc453376f81e None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:42.780730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238a763-6052-4b87-be61-e98be14bf14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:42.781444 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238a763-6052-4b87-be61-e98be14bf14c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:42.781731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238a763-6052-4b87-be61-e98be14bf14c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:42.782009 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238a763-6052-4b87-be61-e98be14bf14c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:42.793538 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238a763-6052-4b87-be61-e98be14bf14c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-2238a763-6052-4b87-be61-e98be14bf14c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:46:42.802363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:42.803559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:42.804567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:42.804567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:42.804567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:42.804567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:42.804567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:46:42.804567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:46:42.804567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:46:42.804567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:46:42.804567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:46:42.804567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:46:42.804567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:46:42.804567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:46:42.804567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 620/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:46:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:42.810391 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f1edf-4f1a-4637-9c57-8045bf95a166 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:42.917685 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f1edf-4f1a-4637-9c57-8045bf95a166 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:42.944406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:42.958250 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d76661-c77b-40f0-8c0b-0510cdf0abf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:42.958965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d76661-c77b-40f0-8c0b-0510cdf0abf6 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:42.959264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d76661-c77b-40f0-8c0b-0510cdf0abf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:42.959545 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d76661-c77b-40f0-8c0b-0510cdf0abf6 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:42.960412 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d76661-c77b-40f0-8c0b-0510cdf0abf6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:42.960852 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d76661-c77b-40f0-8c0b-0510cdf0abf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:42.962421 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d76661-c77b-40f0-8c0b-0510cdf0abf6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:42.971098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 603/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:46:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:46:42.977181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238a763-6052-4b87-be61-e98be14bf14c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:43.025639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:43.083631 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238a763-6052-4b87-be61-e98be14bf14c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:43.117610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c789713-7b8c-4efa-8114-9ad1bce112dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:43.118437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c789713-7b8c-4efa-8114-9ad1bce112dd None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:43.118724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c789713-7b8c-4efa-8114-9ad1bce112dd None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:43.119100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c789713-7b8c-4efa-8114-9ad1bce112dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:43.130581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c789713-7b8c-4efa-8114-9ad1bce112dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:43.132069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c789713-7b8c-4efa-8114-9ad1bce112dd None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-7c789713-7b8c-4efa-8114-9ad1bce112dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:43.150270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:43.151657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:43.152718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:43.152718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:43.152718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:43.152718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:46:43.152718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 621/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:43.157645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d76661-c77b-40f0-8c0b-0510cdf0abf6 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:43.265669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d76661-c77b-40f0-8c0b-0510cdf0abf6 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:43.300408 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e84d78-56d6-4fb4-bf3d-f911c206f338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:43.301089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e84d78-56d6-4fb4-bf3d-f911c206f338 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:43.301378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e84d78-56d6-4fb4-bf3d-f911c206f338 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:43.301660 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e84d78-56d6-4fb4-bf3d-f911c206f338 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:43.302410 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e84d78-56d6-4fb4-bf3d-f911c206f338 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:43.302723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e84d78-56d6-4fb4-bf3d-f911c206f338 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:43.304102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e84d78-56d6-4fb4-bf3d-f911c206f338 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:43.316310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 604/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:43.322285 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c789713-7b8c-4efa-8114-9ad1bce112dd None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:43.419253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:43.426110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c789713-7b8c-4efa-8114-9ad1bce112dd None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:43.461861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb273312-89ac-46b6-9e08-e384cfd70fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:43.462789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb273312-89ac-46b6-9e08-e384cfd70fdc None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:43.464974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb273312-89ac-46b6-9e08-e384cfd70fdc None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:43.464974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb273312-89ac-46b6-9e08-e384cfd70fdc None admin] PATH_INFO: `/v3/users/placement` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:43.471780 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb273312-89ac-46b6-9e08-e384cfd70fdc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:43.473327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb273312-89ac-46b6-9e08-e384cfd70fdc None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-bb273312-89ac-46b6-9e08-e384cfd70fdc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:46:43.489589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:43.490840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:43.491816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:43.491816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:43.491816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:46:43.491816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:46:43.491816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:46:43.491816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:46:43.491816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:46:43.491816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:46:43.491816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:43.491816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:46:43.491816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 622/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:43.496970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e84d78-56d6-4fb4-bf3d-f911c206f338 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:43.548387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:43.606632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e84d78-56d6-4fb4-bf3d-f911c206f338 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:43.648605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89c99c-e036-45cc-b53d-a40f4c66d011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:43.649277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89c99c-e036-45cc-b53d-a40f4c66d011 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:43.649565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89c99c-e036-45cc-b53d-a40f4c66d011 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:43.649843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89c99c-e036-45cc-b53d-a40f4c66d011 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:43.650707 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89c99c-e036-45cc-b53d-a40f4c66d011 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:43.651019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89c99c-e036-45cc-b53d-a40f4c66d011 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:43.652441 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89c99c-e036-45cc-b53d-a40f4c66d011 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:43.678432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 605/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Jul 25 18:46:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:43.678649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb273312-89ac-46b6-9e08-e384cfd70fdc None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:43.782121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb273312-89ac-46b6-9e08-e384cfd70fdc None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:43.818199 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9a084-f6a7-4efc-a3a4-1081eba998f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:43.819030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9a084-f6a7-4efc-a3a4-1081eba998f1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:43.819322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9a084-f6a7-4efc-a3a4-1081eba998f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:43.819630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9a084-f6a7-4efc-a3a4-1081eba998f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:43.831960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9a084-f6a7-4efc-a3a4-1081eba998f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:43.833519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9a084-f6a7-4efc-a3a4-1081eba998f1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-dab9a084-f6a7-4efc-a3a4-1081eba998f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:43.856602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:43.857760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:43.858756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:43.858756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:43.858756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:43.858756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:46:43.858756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 623/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:43.864353 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89c99c-e036-45cc-b53d-a40f4c66d011 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:43.967724 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89c99c-e036-45cc-b53d-a40f4c66d011 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:43.970912 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:44.005786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1bd77-5cde-47f9-a0c1-07d4b356dfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:44.006563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1bd77-5cde-47f9-a0c1-07d4b356dfea None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:44.006855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1bd77-5cde-47f9-a0c1-07d4b356dfea None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:44.007179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1bd77-5cde-47f9-a0c1-07d4b356dfea None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:44.007923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1bd77-5cde-47f9-a0c1-07d4b356dfea None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:44.008235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1bd77-5cde-47f9-a0c1-07d4b356dfea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:44.009623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1bd77-5cde-47f9-a0c1-07d4b356dfea None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:44.024807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 606/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:44.030881 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9a084-f6a7-4efc-a3a4-1081eba998f1 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:44.040615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:44.148251 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9a084-f6a7-4efc-a3a4-1081eba998f1 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:44.185198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4825c94-b69a-4adb-a75f-dbdf8dc287de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:44.186004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4825c94-b69a-4adb-a75f-dbdf8dc287de None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:44.186292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4825c94-b69a-4adb-a75f-dbdf8dc287de None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:44.186608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4825c94-b69a-4adb-a75f-dbdf8dc287de None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:44.194416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4825c94-b69a-4adb-a75f-dbdf8dc287de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:44.195958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4825c94-b69a-4adb-a75f-dbdf8dc287de None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-d4825c94-b69a-4adb-a75f-dbdf8dc287de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:46:44.212914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:44.214054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:44.215286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:46:44.215286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:46:44.215286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:46:44.215286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:46:44.215286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:46:44.215286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:46:44.215286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:44.215286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:46:44.215286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 624/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:46:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:44.236943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1bd77-5cde-47f9-a0c1-07d4b356dfea None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:44.326508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1bd77-5cde-47f9-a0c1-07d4b356dfea None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:44.362680 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8243e-d922-4f4b-82b5-09d99bd025fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:44.363341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8243e-d922-4f4b-82b5-09d99bd025fe None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:44.363626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8243e-d922-4f4b-82b5-09d99bd025fe None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:44.363902 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8243e-d922-4f4b-82b5-09d99bd025fe None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:44.364681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8243e-d922-4f4b-82b5-09d99bd025fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:44.364996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8243e-d922-4f4b-82b5-09d99bd025fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:44.366453 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8243e-d922-4f4b-82b5-09d99bd025fe None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:44.379692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 607/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:46:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:44.386019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4825c94-b69a-4adb-a75f-dbdf8dc287de None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:44.456870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:44.493922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4825c94-b69a-4adb-a75f-dbdf8dc287de None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:44.528092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649aff4-d9ac-455f-83ef-4892aa2fdb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:44.528989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649aff4-d9ac-455f-83ef-4892aa2fdb0c None admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:44.529274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649aff4-d9ac-455f-83ef-4892aa2fdb0c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:44.529551 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649aff4-d9ac-455f-83ef-4892aa2fdb0c None admin] PATH_INFO: `/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/f5027a60da214afda9144b5ae6cce15b/roles/1f308d13b8d04c0b9309a0916dee061f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:44.545528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649aff4-d9ac-455f-83ef-4892aa2fdb0c None admin] RBAC: Authorizing `identity:create_grant(project_id=5d7c3b139f284435aea4ff08a2f0952b, user_id=f5027a60da214afda9144b5ae6cce15b, role_id=1f308d13b8d04c0b9309a0916dee061f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:44.547027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649aff4-d9ac-455f-83ef-4892aa2fdb0c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:44.643549 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 625/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:46:44 2022] PUT /identity/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/f5027a60da214afda9144b5ae6cce15b/roles/1f308d13b8d04c0b9309a0916dee061f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 18:46:46.689984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17525bbc-a1ca-441a-b133-9039a96ec4b2 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:46.690620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17525bbc-a1ca-441a-b133-9039a96ec4b2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:46.690620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17525bbc-a1ca-441a-b133-9039a96ec4b2 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:46.691966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 608/1233] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:46:46.697763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135b6ba-fa6d-4974-b26e-bff377d894e1 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:46.698266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135b6ba-fa6d-4974-b26e-bff377d894e1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:46.698414 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135b6ba-fa6d-4974-b26e-bff377d894e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:46.718678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-e135b6ba-fa6d-4974-b26e-bff377d894e1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:46.720808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e135b6ba-fa6d-4974-b26e-bff377d894e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:46.791283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 626/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:46:46.815526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17525bbc-a1ca-441a-b133-9039a96ec4b2 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:46.845496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:46.937494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17525bbc-a1ca-441a-b133-9039a96ec4b2 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:46.978589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131cf2b-6cab-4c40-80cd-e196cb77d29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:46.979305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131cf2b-6cab-4c40-80cd-e196cb77d29c None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:46.979587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131cf2b-6cab-4c40-80cd-e196cb77d29c None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:46.979864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131cf2b-6cab-4c40-80cd-e196cb77d29c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:46.992277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131cf2b-6cab-4c40-80cd-e196cb77d29c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-9131cf2b-6cab-4c40-80cd-e196cb77d29c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:46:47.000763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:47.001914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:47.002931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:47.002931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:47.002931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:47.002931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:47.002931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:46:47.002931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:46:47.002931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:46:47.002931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:46:47.002931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:46:47.002931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:46:47.002931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 18:46:47.002931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:47.002931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 609/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 18:46:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:47.009059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135b6ba-fa6d-4974-b26e-bff377d894e1 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:47.061001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:47.120111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135b6ba-fa6d-4974-b26e-bff377d894e1 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:47.159616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cd84d-36fb-418b-8532-92428f5df7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:47.160196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cd84d-36fb-418b-8532-92428f5df7cc None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:47.160496 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cd84d-36fb-418b-8532-92428f5df7cc None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:47.160767 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cd84d-36fb-418b-8532-92428f5df7cc None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:47.161526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cd84d-36fb-418b-8532-92428f5df7cc None admin] RBAC: Adding query filter params (name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:47.161842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cd84d-36fb-418b-8532-92428f5df7cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:47.163355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cd84d-36fb-418b-8532-92428f5df7cc None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:47.172197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 627/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 18:46:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:47.178195 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131cf2b-6cab-4c40-80cd-e196cb77d29c None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:47.276016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:47.293954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131cf2b-6cab-4c40-80cd-e196cb77d29c None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:47.337101 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba43e7c-f070-47a8-9869-a5abde519d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:47.337906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba43e7c-f070-47a8-9869-a5abde519d02 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:47.338196 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba43e7c-f070-47a8-9869-a5abde519d02 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:47.338512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba43e7c-f070-47a8-9869-a5abde519d02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:47.349106 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba43e7c-f070-47a8-9869-a5abde519d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:47.350664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba43e7c-f070-47a8-9869-a5abde519d02 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-9ba43e7c-f070-47a8-9869-a5abde519d02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:47.370934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:47.376638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:47.377751 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:47.377751 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:47.377751 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:47.377751 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:47.377751 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 610/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:47.379569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cd84d-36fb-418b-8532-92428f5df7cc None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:47.494435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cd84d-36fb-418b-8532-92428f5df7cc None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:47.504202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:47.530170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b4212-427f-4e8b-91db-d760b33c0f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:47.530758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b4212-427f-4e8b-91db-d760b33c0f36 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:47.531044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b4212-427f-4e8b-91db-d760b33c0f36 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:47.531352 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b4212-427f-4e8b-91db-d760b33c0f36 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:47.532076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b4212-427f-4e8b-91db-d760b33c0f36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:47.532383 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b4212-427f-4e8b-91db-d760b33c0f36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:47.533754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b4212-427f-4e8b-91db-d760b33c0f36 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:47.545720 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 628/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:47.552065 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba43e7c-f070-47a8-9869-a5abde519d02 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:47.669425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba43e7c-f070-47a8-9869-a5abde519d02 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:47.707768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652306c-c11f-4a70-ba93-4d2704b538c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:47.708581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652306c-c11f-4a70-ba93-4d2704b538c4 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:47.708869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652306c-c11f-4a70-ba93-4d2704b538c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:47.709149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652306c-c11f-4a70-ba93-4d2704b538c4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:47.712486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:47.717749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652306c-c11f-4a70-ba93-4d2704b538c4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:47.720023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652306c-c11f-4a70-ba93-4d2704b538c4 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-9652306c-c11f-4a70-ba93-4d2704b538c4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:46:47.737580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:47.738906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:47.740074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:47.740074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:47.740074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:46:47.740074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:46:47.740074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:46:47.740074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:46:47.740074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:46:47.740074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:46:47.740074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:47.740074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:47.740074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 611/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:47.745145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b4212-427f-4e8b-91db-d760b33c0f36 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:47.852219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b4212-427f-4e8b-91db-d760b33c0f36 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:47.887720 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d7eb0-c405-408a-bf55-ce510f9783c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:47.888434 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d7eb0-c405-408a-bf55-ce510f9783c3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:47.888715 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d7eb0-c405-408a-bf55-ce510f9783c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:47.889000 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d7eb0-c405-408a-bf55-ce510f9783c3 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:47.889848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d7eb0-c405-408a-bf55-ce510f9783c3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:47.890164 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d7eb0-c405-408a-bf55-ce510f9783c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:47.892581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d7eb0-c405-408a-bf55-ce510f9783c3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:47.911574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 629/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Jul 25 18:46:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:47.917686 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652306c-c11f-4a70-ba93-4d2704b538c4 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:48.027276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652306c-c11f-4a70-ba93-4d2704b538c4 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:48.065815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976dd589-52be-4fd5-94e7-d0a34e74b188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:48.066516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976dd589-52be-4fd5-94e7-d0a34e74b188 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:48.066802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976dd589-52be-4fd5-94e7-d0a34e74b188 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:48.067084 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976dd589-52be-4fd5-94e7-d0a34e74b188 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:48.134351 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:48.141104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976dd589-52be-4fd5-94e7-d0a34e74b188 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:48.144794 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976dd589-52be-4fd5-94e7-d0a34e74b188 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:48.248570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 612/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:46:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:46:48.255459 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d7eb0-c405-408a-bf55-ce510f9783c3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:48.283261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:48.358263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d7eb0-c405-408a-bf55-ce510f9783c3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:48.395005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3797e1-0313-4b56-a4be-ae4e107300f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:48.395722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3797e1-0313-4b56-a4be-ae4e107300f9 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:48.396007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3797e1-0313-4b56-a4be-ae4e107300f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:48.396318 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3797e1-0313-4b56-a4be-ae4e107300f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:48.407269 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3797e1-0313-4b56-a4be-ae4e107300f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:48.408714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3797e1-0313-4b56-a4be-ae4e107300f9 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-1d3797e1-0313-4b56-a4be-ae4e107300f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:48.428133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:48.429361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:48.430624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:48.430624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:48.430624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:48.430624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:46:48.430624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 630/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:48.435310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976dd589-52be-4fd5-94e7-d0a34e74b188 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:48.533743 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:48.539729 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976dd589-52be-4fd5-94e7-d0a34e74b188 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:48.574731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad975c33-ad29-4be2-9479-f905f0fb7403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:48.575402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad975c33-ad29-4be2-9479-f905f0fb7403 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:48.575693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad975c33-ad29-4be2-9479-f905f0fb7403 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:48.575978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad975c33-ad29-4be2-9479-f905f0fb7403 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:48.576721 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad975c33-ad29-4be2-9479-f905f0fb7403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:48.577033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad975c33-ad29-4be2-9479-f905f0fb7403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:48.578747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad975c33-ad29-4be2-9479-f905f0fb7403 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:48.590545 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 613/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:48.596873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3797e1-0313-4b56-a4be-ae4e107300f9 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:48.681839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:48.703592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3797e1-0313-4b56-a4be-ae4e107300f9 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:48.738872 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53fd83-21c8-4814-8200-a08c9dd40d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:48.744709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53fd83-21c8-4814-8200-a08c9dd40d3d None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:48.744709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53fd83-21c8-4814-8200-a08c9dd40d3d None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:48.744709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53fd83-21c8-4814-8200-a08c9dd40d3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:48.747076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53fd83-21c8-4814-8200-a08c9dd40d3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:48.748622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53fd83-21c8-4814-8200-a08c9dd40d3d None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-aa53fd83-21c8-4814-8200-a08c9dd40d3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:46:48.762795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:48.763921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:48.765812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:46:48.765812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:46:48.765812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:46:48.765812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:46:48.765812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:46:48.765812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:46:48.765812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:48.765812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:46:48.765812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 631/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:46:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:48.771365 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad975c33-ad29-4be2-9479-f905f0fb7403 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:48.879927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad975c33-ad29-4be2-9479-f905f0fb7403 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:48.920005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf941d-aa60-4d86-b971-83fa59812cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:48.920741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf941d-aa60-4d86-b971-83fa59812cde None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:48.921025 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf941d-aa60-4d86-b971-83fa59812cde None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:48.921365 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf941d-aa60-4d86-b971-83fa59812cde None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:48.922205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf941d-aa60-4d86-b971-83fa59812cde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:48.922551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf941d-aa60-4d86-b971-83fa59812cde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:48.924114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf941d-aa60-4d86-b971-83fa59812cde None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:48.953626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 614/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:46:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:46:48.954435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53fd83-21c8-4814-8200-a08c9dd40d3d None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:49.066462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53fd83-21c8-4814-8200-a08c9dd40d3d None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:49.102732 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a40da-8049-440e-b3e7-9bb643ca588f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:49.103402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a40da-8049-440e-b3e7-9bb643ca588f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:49.103693 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a40da-8049-440e-b3e7-9bb643ca588f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:49.103975 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a40da-8049-440e-b3e7-9bb643ca588f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:49.104813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a40da-8049-440e-b3e7-9bb643ca588f None admin] RBAC: Adding query filter params (role.id=1f308d13b8d04c0b9309a0916dee061f, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=f5027a60da214afda9144b5ae6cce15b) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:49.105174 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a40da-8049-440e-b3e7-9bb643ca588f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:49.106597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a40da-8049-440e-b3e7-9bb643ca588f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:49.113241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 632/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:46:48 2022] GET /identity/v3/role_assignments?user.id=f5027a60da214afda9144b5ae6cce15b&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=1f308d13b8d04c0b9309a0916dee061f => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:51.229514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15f47a-5c21-41f8-bc3d-a863fb83ae7d None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:51.230136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15f47a-5c21-41f8-bc3d-a863fb83ae7d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:51.230136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15f47a-5c21-41f8-bc3d-a863fb83ae7d None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:51.231503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 615/1247] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:46:51.238568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fac5ef-cadc-4b29-8e35-930fd6c13624 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:51.244917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fac5ef-cadc-4b29-8e35-930fd6c13624 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:51.244917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fac5ef-cadc-4b29-8e35-930fd6c13624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:51.246470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:51.261040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-09fac5ef-cadc-4b29-8e35-930fd6c13624 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:51.262491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-09fac5ef-cadc-4b29-8e35-930fd6c13624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:51.345863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 633/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:46:51.370829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15f47a-5c21-41f8-bc3d-a863fb83ae7d None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:51.391422 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:51.482435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15f47a-5c21-41f8-bc3d-a863fb83ae7d None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:51.519849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd54b7-da73-4e45-9dbc-29f02bd3b9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:51.521033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd54b7-da73-4e45-9dbc-29f02bd3b9f1 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:51.521483 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd54b7-da73-4e45-9dbc-29f02bd3b9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:51.521893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd54b7-da73-4e45-9dbc-29f02bd3b9f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:51.537681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd54b7-da73-4e45-9dbc-29f02bd3b9f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-dabd54b7-da73-4e45-9dbc-29f02bd3b9f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:46:51.550987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:51.554880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:51.555733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:51.555733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:51.555733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:51.555733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:51.555733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:46:51.555733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:46:51.555733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:46:51.555733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:46:51.555733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:46:51.555733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:46:51.555733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:46:51.555733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:51.555733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 616/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:46:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:51.559134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fac5ef-cadc-4b29-8e35-930fd6c13624 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:51.667215 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fac5ef-cadc-4b29-8e35-930fd6c13624 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:51.697556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:51.703229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23089cf1-bb06-4327-ae34-0def550d320f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:51.703918 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23089cf1-bb06-4327-ae34-0def550d320f None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:51.704203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23089cf1-bb06-4327-ae34-0def550d320f None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:51.704481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23089cf1-bb06-4327-ae34-0def550d320f None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:51.705237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23089cf1-bb06-4327-ae34-0def550d320f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:51.705549 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23089cf1-bb06-4327-ae34-0def550d320f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:51.707088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23089cf1-bb06-4327-ae34-0def550d320f None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:51.716731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 634/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:46:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:51.724004 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd54b7-da73-4e45-9dbc-29f02bd3b9f1 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:51.806773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:51.830828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd54b7-da73-4e45-9dbc-29f02bd3b9f1 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:51.865044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd9458-3c1d-4ca3-a542-ee41e59835a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:51.865849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd9458-3c1d-4ca3-a542-ee41e59835a5 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:51.866135 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd9458-3c1d-4ca3-a542-ee41e59835a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:51.866460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd9458-3c1d-4ca3-a542-ee41e59835a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:51.878170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd9458-3c1d-4ca3-a542-ee41e59835a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:51.879690 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd9458-3c1d-4ca3-a542-ee41e59835a5 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-b5dd9458-3c1d-4ca3-a542-ee41e59835a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:51.897118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:51.899648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:51.900788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:51.900788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:51.900788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:51.900788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:51.900788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 617/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 18:46:51.904162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23089cf1-bb06-4327-ae34-0def550d320f None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:52.007786 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23089cf1-bb06-4327-ae34-0def550d320f None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:52.042665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761bafb-24d9-4858-b380-915ce72f068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:52.043334 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761bafb-24d9-4858-b380-915ce72f068e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:52.043627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761bafb-24d9-4858-b380-915ce72f068e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:52.043946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761bafb-24d9-4858-b380-915ce72f068e None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:52.044661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761bafb-24d9-4858-b380-915ce72f068e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:52.044978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761bafb-24d9-4858-b380-915ce72f068e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:52.046536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761bafb-24d9-4858-b380-915ce72f068e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:52.058471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 635/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:52.064370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd9458-3c1d-4ca3-a542-ee41e59835a5 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:52.171250 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd9458-3c1d-4ca3-a542-ee41e59835a5 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:52.198288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:52.207285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b5a14-b274-4d7c-b5fc-bf2911feae62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:52.208115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b5a14-b274-4d7c-b5fc-bf2911feae62 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:52.208379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b5a14-b274-4d7c-b5fc-bf2911feae62 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:52.208658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b5a14-b274-4d7c-b5fc-bf2911feae62 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:52.216853 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b5a14-b274-4d7c-b5fc-bf2911feae62 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:52.218597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b5a14-b274-4d7c-b5fc-bf2911feae62 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-475b5a14-b274-4d7c-b5fc-bf2911feae62 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:46:52.234032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:52.235583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:52.236542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:52.236542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:52.236542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:46:52.236542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:46:52.236542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:46:52.236542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:46:52.236542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:46:52.236542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:46:52.236542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:52.236542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:52.236542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 618/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:52.241181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761bafb-24d9-4858-b380-915ce72f068e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:52.299650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:52.346652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761bafb-24d9-4858-b380-915ce72f068e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:52.384131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442936be-0818-4c32-b2bf-51b424d21c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:52.384832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442936be-0818-4c32-b2bf-51b424d21c1c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:52.387157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442936be-0818-4c32-b2bf-51b424d21c1c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:52.387157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442936be-0818-4c32-b2bf-51b424d21c1c None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:52.387157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442936be-0818-4c32-b2bf-51b424d21c1c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:52.387157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442936be-0818-4c32-b2bf-51b424d21c1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:52.388466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442936be-0818-4c32-b2bf-51b424d21c1c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:52.407440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 636/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Jul 25 18:46:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:52.413240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b5a14-b274-4d7c-b5fc-bf2911feae62 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:52.524019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b5a14-b274-4d7c-b5fc-bf2911feae62 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:52.562278 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e3761-92ad-47ac-8ebc-da1118622292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:52.562961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e3761-92ad-47ac-8ebc-da1118622292 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:52.563247 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e3761-92ad-47ac-8ebc-da1118622292 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:52.563527 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e3761-92ad-47ac-8ebc-da1118622292 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:52.624551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:52.638272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e3761-92ad-47ac-8ebc-da1118622292 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:52.639867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e3761-92ad-47ac-8ebc-da1118622292 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:52.743624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 619/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:46:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:46:52.750861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442936be-0818-4c32-b2bf-51b424d21c1c None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:52.864471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442936be-0818-4c32-b2bf-51b424d21c1c None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:52.882648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:52.903191 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2520-de62-4253-8e12-66461842ea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:52.904000 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2520-de62-4253-8e12-66461842ea2e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:52.904290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2520-de62-4253-8e12-66461842ea2e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:52.904572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2520-de62-4253-8e12-66461842ea2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:52.916639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e2520-de62-4253-8e12-66461842ea2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:52.918318 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e2520-de62-4253-8e12-66461842ea2e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-3e3e2520-de62-4253-8e12-66461842ea2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:52.938927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:52.944905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:52.946078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:52.946078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:52.946078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:52.946078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:46:52.946078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 637/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:52.947911 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e3761-92ad-47ac-8ebc-da1118622292 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:53.050244 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:53.062695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e3761-92ad-47ac-8ebc-da1118622292 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:53.102875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fe1df-f2bf-4eca-bec3-05d637d31764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:53.103532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fe1df-f2bf-4eca-bec3-05d637d31764 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:53.103978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fe1df-f2bf-4eca-bec3-05d637d31764 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:53.104101 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fe1df-f2bf-4eca-bec3-05d637d31764 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:53.104827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fe1df-f2bf-4eca-bec3-05d637d31764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:53.105133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fe1df-f2bf-4eca-bec3-05d637d31764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:53.106574 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fe1df-f2bf-4eca-bec3-05d637d31764 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:53.119769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 620/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:53.126090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2520-de62-4253-8e12-66461842ea2e None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:53.229572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2520-de62-4253-8e12-66461842ea2e None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:53.267373 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32440dc-8bfe-4cdb-bc43-9e6306924dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:53.268586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32440dc-8bfe-4cdb-bc43-9e6306924dbf None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:53.268953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32440dc-8bfe-4cdb-bc43-9e6306924dbf None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:53.269320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32440dc-8bfe-4cdb-bc43-9e6306924dbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:53.276325 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32440dc-8bfe-4cdb-bc43-9e6306924dbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:53.277797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32440dc-8bfe-4cdb-bc43-9e6306924dbf None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-a32440dc-8bfe-4cdb-bc43-9e6306924dbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:46:53.292214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:53.293340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:53.294346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:46:53.294346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:46:53.294346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:46:53.294346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:46:53.294346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:46:53.294346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:46:53.294346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:53.294346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:46:53.294346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 638/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:46:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:53.299806 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fe1df-f2bf-4eca-bec3-05d637d31764 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:53.408005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fe1df-f2bf-4eca-bec3-05d637d31764 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:53.442540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12cf8e-38ae-43a6-a7aa-326775d6b1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:53.443212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12cf8e-38ae-43a6-a7aa-326775d6b1ea None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:53.443560 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12cf8e-38ae-43a6-a7aa-326775d6b1ea None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:53.443843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12cf8e-38ae-43a6-a7aa-326775d6b1ea None admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:53.444635 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12cf8e-38ae-43a6-a7aa-326775d6b1ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:53.444967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12cf8e-38ae-43a6-a7aa-326775d6b1ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:53.446420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12cf8e-38ae-43a6-a7aa-326775d6b1ea None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:53.461000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 621/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:46:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:53.467209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32440dc-8bfe-4cdb-bc43-9e6306924dbf None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:53.494567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:53.567870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32440dc-8bfe-4cdb-bc43-9e6306924dbf None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:53.608275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6683b-5819-40a2-ac77-7ebdafe86979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:53.608962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6683b-5819-40a2-ac77-7ebdafe86979 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:53.609245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6683b-5819-40a2-ac77-7ebdafe86979 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:53.609522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6683b-5819-40a2-ac77-7ebdafe86979 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:53.610388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6683b-5819-40a2-ac77-7ebdafe86979 None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=f5027a60da214afda9144b5ae6cce15b) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:53.610714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6683b-5819-40a2-ac77-7ebdafe86979 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:53.612176 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6683b-5819-40a2-ac77-7ebdafe86979 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:53.618558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 639/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:46:53 2022] GET /identity/v3/role_assignments?user.id=f5027a60da214afda9144b5ae6cce15b&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:55.654669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b3527-f5cf-4f50-8aca-9a7cb63366a8 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:55.655364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b3527-f5cf-4f50-8aca-9a7cb63366a8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:55.655364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b3527-f5cf-4f50-8aca-9a7cb63366a8 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:55.656633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 622/1261] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:46:55.661908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321b125-25b6-4ad0-b72b-5567ab4528a3 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:55.662344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321b125-25b6-4ad0-b72b-5567ab4528a3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:55.662667 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321b125-25b6-4ad0-b72b-5567ab4528a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:55.681081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-c321b125-25b6-4ad0-b72b-5567ab4528a3 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:55.682275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c321b125-25b6-4ad0-b72b-5567ab4528a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:55.751055 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 640/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:46:55.776242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b3527-f5cf-4f50-8aca-9a7cb63366a8 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:55.796129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:55.891583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b3527-f5cf-4f50-8aca-9a7cb63366a8 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:55.926089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f408d-c2c1-4004-9694-2b4d6e63b173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:55.926822 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f408d-c2c1-4004-9694-2b4d6e63b173 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:55.927106 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f408d-c2c1-4004-9694-2b4d6e63b173 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:55.927375 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f408d-c2c1-4004-9694-2b4d6e63b173 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:55.937820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f408d-c2c1-4004-9694-2b4d6e63b173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-073f408d-c2c1-4004-9694-2b4d6e63b173 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:46:55.945612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:55.946631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:55.947622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:55.947622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:55.947622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:55.947622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:55.947622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:46:55.947622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:46:55.947622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:46:55.947622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:46:55.947622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:46:55.947622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:46:55.947622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:46:55.947622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:55.947622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 623/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:46:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:55.953596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321b125-25b6-4ad0-b72b-5567ab4528a3 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:55.972522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:56.053516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321b125-25b6-4ad0-b72b-5567ab4528a3 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:56.088518 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f54f4d-08c8-4692-b240-ecc79cfe2594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:56.089195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f54f4d-08c8-4692-b240-ecc79cfe2594 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:56.089483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f54f4d-08c8-4692-b240-ecc79cfe2594 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:56.089763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f54f4d-08c8-4692-b240-ecc79cfe2594 None admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:56.090536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f54f4d-08c8-4692-b240-ecc79cfe2594 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:56.090851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f54f4d-08c8-4692-b240-ecc79cfe2594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:56.092733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f54f4d-08c8-4692-b240-ecc79cfe2594 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:56.101180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 641/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:46:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:56.107693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f408d-c2c1-4004-9694-2b4d6e63b173 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:56.180827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:56.215100 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f408d-c2c1-4004-9694-2b4d6e63b173 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:56.249839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc057b0-9510-4606-9f08-e998a0ee8dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:56.250665 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc057b0-9510-4606-9f08-e998a0ee8dba None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:56.250973 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc057b0-9510-4606-9f08-e998a0ee8dba None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:56.251255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc057b0-9510-4606-9f08-e998a0ee8dba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:56.262317 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc057b0-9510-4606-9f08-e998a0ee8dba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:56.263901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc057b0-9510-4606-9f08-e998a0ee8dba None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-2fc057b0-9510-4606-9f08-e998a0ee8dba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:56.282658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:56.283876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:56.285141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:56.285141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:56.285141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:56.285141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:56.285141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 624/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:56.289781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f54f4d-08c8-4692-b240-ecc79cfe2594 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:56.372275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:56.392268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f54f4d-08c8-4692-b240-ecc79cfe2594 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:56.426603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8243d0-a1ec-450d-8393-49be86efef0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:56.427276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8243d0-a1ec-450d-8393-49be86efef0a None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:56.427563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8243d0-a1ec-450d-8393-49be86efef0a None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:56.427850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8243d0-a1ec-450d-8393-49be86efef0a None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:56.428573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8243d0-a1ec-450d-8393-49be86efef0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:56.428885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8243d0-a1ec-450d-8393-49be86efef0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:56.430263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8243d0-a1ec-450d-8393-49be86efef0a None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:56.441848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 642/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:56.447829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc057b0-9510-4606-9f08-e998a0ee8dba None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:56.551726 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc057b0-9510-4606-9f08-e998a0ee8dba None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:56.571316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:56.586079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1c4e9-fb1e-4d6a-b27c-8ff4573774f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:56.586905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1c4e9-fb1e-4d6a-b27c-8ff4573774f6 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:56.587192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1c4e9-fb1e-4d6a-b27c-8ff4573774f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:56.587539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1c4e9-fb1e-4d6a-b27c-8ff4573774f6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:56.595850 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1c4e9-fb1e-4d6a-b27c-8ff4573774f6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:56.597288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1c4e9-fb1e-4d6a-b27c-8ff4573774f6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-7cb1c4e9-fb1e-4d6a-b27c-8ff4573774f6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:46:56.613149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:56.614424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:56.619964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:56.619964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:56.619964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:46:56.619964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:46:56.619964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:46:56.619964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:46:56.619964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:46:56.619964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:46:56.619964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:46:56.619964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:56.619964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 625/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:56.620749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8243d0-a1ec-450d-8393-49be86efef0a None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:56.724059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8243d0-a1ec-450d-8393-49be86efef0a None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:56.757596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b64f9-5c8f-430b-be5a-1bc2bff9bfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:56.758272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b64f9-5c8f-430b-be5a-1bc2bff9bfc1 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:56.758583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b64f9-5c8f-430b-be5a-1bc2bff9bfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:56.758860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b64f9-5c8f-430b-be5a-1bc2bff9bfc1 None admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:56.759702 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b64f9-5c8f-430b-be5a-1bc2bff9bfc1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:56.760010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b64f9-5c8f-430b-be5a-1bc2bff9bfc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:56.761441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b64f9-5c8f-430b-be5a-1bc2bff9bfc1 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:56.772154 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:56.780652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 643/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Jul 25 18:46:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:56.786528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1c4e9-fb1e-4d6a-b27c-8ff4573774f6 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:56.890679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1c4e9-fb1e-4d6a-b27c-8ff4573774f6 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:56.926503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cefb9b-9e2f-4154-9c2c-23d0dcd0359a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:56.927303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cefb9b-9e2f-4154-9c2c-23d0dcd0359a None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:56.927592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cefb9b-9e2f-4154-9c2c-23d0dcd0359a None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:56.927978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cefb9b-9e2f-4154-9c2c-23d0dcd0359a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:56.940314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cefb9b-9e2f-4154-9c2c-23d0dcd0359a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:56.941771 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cefb9b-9e2f-4154-9c2c-23d0dcd0359a None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-e1cefb9b-9e2f-4154-9c2c-23d0dcd0359a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:46:56.959600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:56.960765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:56.962158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:46:56.962158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:46:56.962158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:46:56.962158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:56.962158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 626/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:56.967101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b64f9-5c8f-430b-be5a-1bc2bff9bfc1 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:57.070766 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b64f9-5c8f-430b-be5a-1bc2bff9bfc1 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:57.107054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ce704-3e09-4d35-a375-dda954c34af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:57.108352 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ce704-3e09-4d35-a375-dda954c34af3 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:57.108641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ce704-3e09-4d35-a375-dda954c34af3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:57.109038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ce704-3e09-4d35-a375-dda954c34af3 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:57.109831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ce704-3e09-4d35-a375-dda954c34af3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:57.110138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ce704-3e09-4d35-a375-dda954c34af3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:57.111553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ce704-3e09-4d35-a375-dda954c34af3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:57.124774 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 644/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 18:46:57.130831 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cefb9b-9e2f-4154-9c2c-23d0dcd0359a None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:57.164432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:57.235234 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cefb9b-9e2f-4154-9c2c-23d0dcd0359a None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:57.270551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389051c-58e9-43e3-b79d-75e581aa4299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:57.271660 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389051c-58e9-43e3-b79d-75e581aa4299 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:57.272147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389051c-58e9-43e3-b79d-75e581aa4299 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:57.273302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389051c-58e9-43e3-b79d-75e581aa4299 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:57.280361 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389051c-58e9-43e3-b79d-75e581aa4299 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:57.281830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389051c-58e9-43e3-b79d-75e581aa4299 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-6389051c-58e9-43e3-b79d-75e581aa4299 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:46:57.296162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:46:57.297149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:46:57.298122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:46:57.298122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:46:57.298122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:46:57.298122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:46:57.298122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:46:57.298122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:46:57.298122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:46:57.298122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:46:57.298122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 627/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:46:57.303546 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ce704-3e09-4d35-a375-dda954c34af3 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:57.366596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:57.410597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ce704-3e09-4d35-a375-dda954c34af3 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:57.445391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44790e-d4dc-4967-8217-db763622b62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:57.446067 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44790e-d4dc-4967-8217-db763622b62c None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:57.446391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44790e-d4dc-4967-8217-db763622b62c None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:57.446710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44790e-d4dc-4967-8217-db763622b62c None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:57.454713 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44790e-d4dc-4967-8217-db763622b62c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:46:57.454713 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44790e-d4dc-4967-8217-db763622b62c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:57.454713 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44790e-d4dc-4967-8217-db763622b62c None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:57.462319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 645/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:46:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:46:57.469267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389051c-58e9-43e3-b79d-75e581aa4299 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:46:57.567368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:46:57.576286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389051c-58e9-43e3-b79d-75e581aa4299 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:46:57.612478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef68617-77fa-4b8a-8cb5-c6ca1e8c1cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:46:57.613440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef68617-77fa-4b8a-8cb5-c6ca1e8c1cb2 None admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:57.613719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef68617-77fa-4b8a-8cb5-c6ca1e8c1cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:57.613997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef68617-77fa-4b8a-8cb5-c6ca1e8c1cb2 None admin] PATH_INFO: `/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/f5027a60da214afda9144b5ae6cce15b/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:57.616023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef68617-77fa-4b8a-8cb5-c6ca1e8c1cb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d7c3b139f284435aea4ff08a2f0952b, user_id=f5027a60da214afda9144b5ae6cce15b, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:46:57.617514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef68617-77fa-4b8a-8cb5-c6ca1e8c1cb2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:46:57.758121 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 628/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 18:46:57 2022] PUT /identity/v3/projects/5d7c3b139f284435aea4ff08a2f0952b/users/f5027a60da214afda9144b5ae6cce15b/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 18:46:59.815044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214c88e-318a-4f45-8821-a55d785f76d3 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:59.816548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214c88e-318a-4f45-8821-a55d785f76d3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:59.816548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214c88e-318a-4f45-8821-a55d785f76d3 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:59.817030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 646/1274] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:46:59.822749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf76bc6-ce06-47e2-98e9-499ec322f9a0 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:46:59.823312 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf76bc6-ce06-47e2-98e9-499ec322f9a0 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:46:59.823312 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf76bc6-ce06-47e2-98e9-499ec322f9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:46:59.844357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-5bf76bc6-ce06-47e2-98e9-499ec322f9a0 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:46:59.849243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5bf76bc6-ce06-47e2-98e9-499ec322f9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:46:59.915381 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 629/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:46:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:46:59.939585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214c88e-318a-4f45-8821-a55d785f76d3 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:00.043435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214c88e-318a-4f45-8821-a55d785f76d3 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:00.062079 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:00.083382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39609abe-1076-4d51-9a1e-a2f0cbed5805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:00.084113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39609abe-1076-4d51-9a1e-a2f0cbed5805 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:00.084398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39609abe-1076-4d51-9a1e-a2f0cbed5805 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:00.084674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39609abe-1076-4d51-9a1e-a2f0cbed5805 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:00.096848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39609abe-1076-4d51-9a1e-a2f0cbed5805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-39609abe-1076-4d51-9a1e-a2f0cbed5805 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 18:47:00.105038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:00.106048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:00.107699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:00.107699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:00.107699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:00.107699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:00.107699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 18:47:00.107699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 18:47:00.107699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 18:47:00.107699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 18:47:00.107699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 18:47:00.107699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 18:47:00.107699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 18:47:00.107699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:47:00.107699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 647/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 18:46:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:00.113964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf76bc6-ce06-47e2-98e9-499ec322f9a0 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:00.199844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:00.219581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf76bc6-ce06-47e2-98e9-499ec322f9a0 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:00.255775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ef083-df6d-4936-9a13-1f8cf38810f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:00.256479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ef083-df6d-4936-9a13-1f8cf38810f2 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:00.256734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ef083-df6d-4936-9a13-1f8cf38810f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:00.257016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ef083-df6d-4936-9a13-1f8cf38810f2 None admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:00.257784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ef083-df6d-4936-9a13-1f8cf38810f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:00.258098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ef083-df6d-4936-9a13-1f8cf38810f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:00.259497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ef083-df6d-4936-9a13-1f8cf38810f2 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:00.268366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 630/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 18:47:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:00.274416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39609abe-1076-4d51-9a1e-a2f0cbed5805 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:00.381230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39609abe-1076-4d51-9a1e-a2f0cbed5805 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:00.417248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c85c2-4931-4ab0-963f-a481778223c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:00.418135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c85c2-4931-4ab0-963f-a481778223c8 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:00.418465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c85c2-4931-4ab0-963f-a481778223c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:00.418748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c85c2-4931-4ab0-963f-a481778223c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:00.430536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c85c2-4931-4ab0-963f-a481778223c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:00.432026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c85c2-4931-4ab0-963f-a481778223c8 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-917c85c2-4931-4ab0-963f-a481778223c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:47:00.451289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:00.452295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:00.453611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:47:00.453611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:47:00.453611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:47:00.453611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:47:00.453611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 648/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:00.459866 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ef083-df6d-4936-9a13-1f8cf38810f2 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:00.567831 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ef083-df6d-4936-9a13-1f8cf38810f2 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:00.602340 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ce0b7-2e74-4241-94da-d0bd6e50b1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:00.603039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ce0b7-2e74-4241-94da-d0bd6e50b1e1 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:00.603328 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ce0b7-2e74-4241-94da-d0bd6e50b1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:00.603615 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ce0b7-2e74-4241-94da-d0bd6e50b1e1 None admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:00.604349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ce0b7-2e74-4241-94da-d0bd6e50b1e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:00.604659 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ce0b7-2e74-4241-94da-d0bd6e50b1e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:00.606035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ce0b7-2e74-4241-94da-d0bd6e50b1e1 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:00.618942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 631/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:00.625891 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c85c2-4931-4ab0-963f-a481778223c8 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:00.660340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:00.731059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c85c2-4931-4ab0-963f-a481778223c8 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:00.767816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd0a85-002b-4c4c-9559-3d130f8c5bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:00.768637 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0a85-002b-4c4c-9559-3d130f8c5bee None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:00.768922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0a85-002b-4c4c-9559-3d130f8c5bee None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:00.769210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0a85-002b-4c4c-9559-3d130f8c5bee None admin] PATH_INFO: `/v3/users/placement` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:00.776624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd0a85-002b-4c4c-9559-3d130f8c5bee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:00.778062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd0a85-002b-4c4c-9559-3d130f8c5bee None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-b4cd0a85-002b-4c4c-9559-3d130f8c5bee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 18:47:00.792818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:00.794995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:00.796415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:00.796415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:00.796415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 18:47:00.796415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 18:47:00.796415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 18:47:00.796415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 18:47:00.796415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 18:47:00.796415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 18:47:00.796415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 18:47:00.796415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:47:00.796415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 649/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:47:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:00.800568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ce0b7-2e74-4241-94da-d0bd6e50b1e1 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:00.804874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:00.906337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ce0b7-2e74-4241-94da-d0bd6e50b1e1 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:00.940793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f944c-262f-43f8-beff-551b0ed45fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:00.941484 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f944c-262f-43f8-beff-551b0ed45fc0 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:00.949279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f944c-262f-43f8-beff-551b0ed45fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:00.949279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f944c-262f-43f8-beff-551b0ed45fc0 None admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:00.949279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f944c-262f-43f8-beff-551b0ed45fc0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:00.949279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f944c-262f-43f8-beff-551b0ed45fc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:00.949279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f944c-262f-43f8-beff-551b0ed45fc0 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:00.963356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 632/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Jul 25 18:47:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:00.969982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd0a85-002b-4c4c-9559-3d130f8c5bee None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:01.064652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:01.074237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd0a85-002b-4c4c-9559-3d130f8c5bee None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:01.110053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb638-6174-4e8d-b5ea-927eb564c58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:01.110756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb638-6174-4e8d-b5ea-927eb564c58b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:01.111046 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb638-6174-4e8d-b5ea-927eb564c58b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:01.111329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb638-6174-4e8d-b5ea-927eb564c58b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:01.183246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bb638-6174-4e8d-b5ea-927eb564c58b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:01.184769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bb638-6174-4e8d-b5ea-927eb564c58b None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:01.281070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:01.291173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 650/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 18:47:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 18:47:01.298210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f944c-262f-43f8-beff-551b0ed45fc0 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:01.362653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:01.408064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f944c-262f-43f8-beff-551b0ed45fc0 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:01.447631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0980457-269f-483e-b729-442ead4a76c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:01.448437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0980457-269f-483e-b729-442ead4a76c6 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:01.448976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0980457-269f-483e-b729-442ead4a76c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:01.449319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0980457-269f-483e-b729-442ead4a76c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:01.465399 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0980457-269f-483e-b729-442ead4a76c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:01.468034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0980457-269f-483e-b729-442ead4a76c6 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-f0980457-269f-483e-b729-442ead4a76c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 18:47:01.487855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:01.488893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:01.490055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 18:47:01.490055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 18:47:01.490055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 18:47:01.490055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:47:01.490055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 633/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:01.494964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb638-6174-4e8d-b5ea-927eb564c58b None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:01.596756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb638-6174-4e8d-b5ea-927eb564c58b None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:01.631759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee19838-830b-44b5-ac45-7ab48a77b9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:01.632429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee19838-830b-44b5-ac45-7ab48a77b9c4 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:01.632714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee19838-830b-44b5-ac45-7ab48a77b9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:01.632991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee19838-830b-44b5-ac45-7ab48a77b9c4 None admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:01.633705 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee19838-830b-44b5-ac45-7ab48a77b9c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:01.634018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee19838-830b-44b5-ac45-7ab48a77b9c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:01.635540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee19838-830b-44b5-ac45-7ab48a77b9c4 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:01.647299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 651/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 18:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:01.653670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0980457-269f-483e-b729-442ead4a76c6 None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:01.763637 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0980457-269f-483e-b729-442ead4a76c6 None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:01.778900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:01.799720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f7ede-7394-4a77-93ee-61f2bf50961f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:01.800514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f7ede-7394-4a77-93ee-61f2bf50961f None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:01.800796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f7ede-7394-4a77-93ee-61f2bf50961f None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:01.801068 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f7ede-7394-4a77-93ee-61f2bf50961f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:01.807977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f7ede-7394-4a77-93ee-61f2bf50961f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:01.810048 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f7ede-7394-4a77-93ee-61f2bf50961f None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application [None req-eb9f7ede-7394-4a77-93ee-61f2bf50961f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 18:47:01.824984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:01.826034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:01.826968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 18:47:01.826968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 18:47:01.826968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 18:47:01.826968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 18:47:01.826968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 18:47:01.826968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 18:47:01.826968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 18:47:01.826968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: ERROR keystone.server.flask.application Jul 25 18:47:01.826968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 634/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 18:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:01.832987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee19838-830b-44b5-ac45-7ab48a77b9c4 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:01.887751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:01.940245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee19838-830b-44b5-ac45-7ab48a77b9c4 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:01.973519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ed7c8-e422-4f52-b821-f3af13bdf46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:01.974164 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ed7c8-e422-4f52-b821-f3af13bdf46e None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:01.974447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ed7c8-e422-4f52-b821-f3af13bdf46e None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:01.974730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ed7c8-e422-4f52-b821-f3af13bdf46e None admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:01.975512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ed7c8-e422-4f52-b821-f3af13bdf46e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:01.975824 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ed7c8-e422-4f52-b821-f3af13bdf46e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:01.977245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ed7c8-e422-4f52-b821-f3af13bdf46e None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:01.989697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 652/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 18:47:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:01.995903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f7ede-7394-4a77-93ee-61f2bf50961f None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:02.097769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f7ede-7394-4a77-93ee-61f2bf50961f None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:02.134530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e87da-5dd0-45ca-ad00-d9b44f057be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:02.135210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e87da-5dd0-45ca-ad00-d9b44f057be7 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:02.135502 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e87da-5dd0-45ca-ad00-d9b44f057be7 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:02.135785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e87da-5dd0-45ca-ad00-d9b44f057be7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:02.136626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e87da-5dd0-45ca-ad00-d9b44f057be7 None admin] RBAC: Adding query filter params (role.id=9bfe07ad221449c586c8e262f86d7f48, scope.project.id=5d7c3b139f284435aea4ff08a2f0952b, user.id=f5027a60da214afda9144b5ae6cce15b) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:02.136948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e87da-5dd0-45ca-ad00-d9b44f057be7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:02.139381 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e87da-5dd0-45ca-ad00-d9b44f057be7 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:02.145689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 635/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 18:47:01 2022] GET /identity/v3/role_assignments?user.id=f5027a60da214afda9144b5ae6cce15b&scope.project.id=5d7c3b139f284435aea4ff08a2f0952b&role.id=9bfe07ad221449c586c8e262f86d7f48 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:04.207330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf72235-0944-4ba2-8f1d-c51aa2dc6917 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:04.207911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf72235-0944-4ba2-8f1d-c51aa2dc6917 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:04.208009 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf72235-0944-4ba2-8f1d-c51aa2dc6917 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:04.209329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 653/1288] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:47:04.214759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da6113-89ae-4975-bc10-9fc6cfca36c8 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:04.215922 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da6113-89ae-4975-bc10-9fc6cfca36c8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:04.215922 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da6113-89ae-4975-bc10-9fc6cfca36c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:04.235648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-80da6113-89ae-4975-bc10-9fc6cfca36c8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:47:04.239016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-80da6113-89ae-4975-bc10-9fc6cfca36c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:47:04.270298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:04.311544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 636/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:47:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:47:04.334431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf72235-0944-4ba2-8f1d-c51aa2dc6917 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:04.442822 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf72235-0944-4ba2-8f1d-c51aa2dc6917 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:04.460222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:04.488581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb896d69-00a1-4092-ae38-f8a7a3e3bc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:04.489396 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb896d69-00a1-4092-ae38-f8a7a3e3bc9b None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:04.489686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb896d69-00a1-4092-ae38-f8a7a3e3bc9b None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:04.489969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb896d69-00a1-4092-ae38-f8a7a3e3bc9b None admin] PATH_INFO: `/v3/services/placement` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:04.490662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb896d69-00a1-4092-ae38-f8a7a3e3bc9b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:04.492128 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb896d69-00a1-4092-ae38-f8a7a3e3bc9b None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-bb896d69-00a1-4092-ae38-f8a7a3e3bc9b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:04.505038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:04.506408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:47:04.507440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 654/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 18:47:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:04.512563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da6113-89ae-4975-bc10-9fc6cfca36c8 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:04.617190 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da6113-89ae-4975-bc10-9fc6cfca36c8 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:04.653503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c5eaa-f4ac-4f72-81ca-e75c3a2f8319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:04.654175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c5eaa-f4ac-4f72-81ca-e75c3a2f8319 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:04.654494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c5eaa-f4ac-4f72-81ca-e75c3a2f8319 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:04.654780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c5eaa-f4ac-4f72-81ca-e75c3a2f8319 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:04.655505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c5eaa-f4ac-4f72-81ca-e75c3a2f8319 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:04.655815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c5eaa-f4ac-4f72-81ca-e75c3a2f8319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:04.657186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c5eaa-f4ac-4f72-81ca-e75c3a2f8319 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:04.662349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 637/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Jul 25 18:47:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:04.669247 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb896d69-00a1-4092-ae38-f8a7a3e3bc9b None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:04.775489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb896d69-00a1-4092-ae38-f8a7a3e3bc9b None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:04.815038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15550369-fd6e-4f7b-bcfe-7cb9897c6188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:04.815683 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15550369-fd6e-4f7b-bcfe-7cb9897c6188 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:04.815976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15550369-fd6e-4f7b-bcfe-7cb9897c6188 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:04.816255 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15550369-fd6e-4f7b-bcfe-7cb9897c6188 None admin] PATH_INFO: `/v3/services` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:04.817058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15550369-fd6e-4f7b-bcfe-7cb9897c6188 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:04.817371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15550369-fd6e-4f7b-bcfe-7cb9897c6188 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:04.818779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15550369-fd6e-4f7b-bcfe-7cb9897c6188 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:04.824155 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 655/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Jul 25 18:47:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:06.888644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b520131-afba-4a5a-95c1-f98026f8a8a9 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:06.889200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b520131-afba-4a5a-95c1-f98026f8a8a9 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:06.889200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b520131-afba-4a5a-95c1-f98026f8a8a9 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:06.890672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 638/1293] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:47:06.896020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad961dd6-98e1-463f-94f2-63a07385bcac None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:06.896426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad961dd6-98e1-463f-94f2-63a07385bcac None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:06.896591 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad961dd6-98e1-463f-94f2-63a07385bcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:06.915563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ad961dd6-98e1-463f-94f2-63a07385bcac None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:47:06.916767 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ad961dd6-98e1-463f-94f2-63a07385bcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:47:06.950555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:06.992803 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 656/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:47:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:47:07.017040 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b520131-afba-4a5a-95c1-f98026f8a8a9 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:07.054515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:07.134878 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b520131-afba-4a5a-95c1-f98026f8a8a9 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:07.173933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80375d62-8812-4758-a927-87b1201b4111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:07.175331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80375d62-8812-4758-a927-87b1201b4111 None admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:07.175331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80375d62-8812-4758-a927-87b1201b4111 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:07.175331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80375d62-8812-4758-a927-87b1201b4111 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:07.175962 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80375d62-8812-4758-a927-87b1201b4111 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:07.177341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80375d62-8812-4758-a927-87b1201b4111 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:07.364164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 639/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 18:47:07 2022] POST /identity/v3/services => generated 246 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 18:47:09.409855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49307e-0b8b-4e25-8145-ccafbb699585 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:09.410710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49307e-0b8b-4e25-8145-ccafbb699585 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:09.410710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49307e-0b8b-4e25-8145-ccafbb699585 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:09.411826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 657/1296] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:47:09.418716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8445a-46a0-494d-8d95-9dac7af32105 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:09.420947 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8445a-46a0-494d-8d95-9dac7af32105 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:09.420947 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8445a-46a0-494d-8d95-9dac7af32105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:09.438354 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-f2b8445a-46a0-494d-8d95-9dac7af32105 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:47:09.439673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f2b8445a-46a0-494d-8d95-9dac7af32105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:47:09.519047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 640/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:47:09 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:47:09.543855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49307e-0b8b-4e25-8145-ccafbb699585 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:09.648922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49307e-0b8b-4e25-8145-ccafbb699585 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:09.685309 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298576d4-9784-4e08-8739-b96ac92fd0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:09.686212 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298576d4-9784-4e08-8739-b96ac92fd0d5 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:09.686544 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298576d4-9784-4e08-8739-b96ac92fd0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:09.686814 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298576d4-9784-4e08-8739-b96ac92fd0d5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:09.687472 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298576d4-9784-4e08-8739-b96ac92fd0d5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:09.688850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298576d4-9784-4e08-8739-b96ac92fd0d5 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-298576d4-9784-4e08-8739-b96ac92fd0d5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:09.702287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:09.703600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:47:09.710995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 658/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 18:47:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:09.712065 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8445a-46a0-494d-8d95-9dac7af32105 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:09.730585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:09.814208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8445a-46a0-494d-8d95-9dac7af32105 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:09.848828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d04c2-8bfd-44ee-9e15-a1c4975dd12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:09.849498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d04c2-8bfd-44ee-9e15-a1c4975dd12c None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:09.849832 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d04c2-8bfd-44ee-9e15-a1c4975dd12c None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:09.850117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d04c2-8bfd-44ee-9e15-a1c4975dd12c None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:09.850857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d04c2-8bfd-44ee-9e15-a1c4975dd12c None admin] RBAC: Adding query filter params (name=placement) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:09.851172 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d04c2-8bfd-44ee-9e15-a1c4975dd12c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:09.852539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d04c2-8bfd-44ee-9e15-a1c4975dd12c None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:09.858367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 641/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Jul 25 18:47:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:09.864497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298576d4-9784-4e08-8739-b96ac92fd0d5 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:09.897043 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:09.967630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298576d4-9784-4e08-8739-b96ac92fd0d5 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:10.004828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31fe78-712d-46f0-a005-49285506f392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:10.005493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31fe78-712d-46f0-a005-49285506f392 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:10.005779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31fe78-712d-46f0-a005-49285506f392 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:10.006060 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31fe78-712d-46f0-a005-49285506f392 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:10.006874 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31fe78-712d-46f0-a005-49285506f392 None admin] RBAC: Adding query filter params (interface=public, service_id=29ebaa94027c4990a1beb08888333e35, region_id=RegionOne) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:10.007188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31fe78-712d-46f0-a005-49285506f392 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:10.008575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31fe78-712d-46f0-a005-49285506f392 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:10.014391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 659/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 18:47:09 2022] GET /identity/v3/endpoints?service_id=29ebaa94027c4990a1beb08888333e35&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:10.020409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d04c2-8bfd-44ee-9e15-a1c4975dd12c None admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:10.106715 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:10.125521 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d04c2-8bfd-44ee-9e15-a1c4975dd12c None admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:10.161961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876e09e-e7d3-4b17-9675-85b3fd34d2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:10.162677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876e09e-e7d3-4b17-9675-85b3fd34d2e8 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:10.162965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876e09e-e7d3-4b17-9675-85b3fd34d2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:10.163244 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876e09e-e7d3-4b17-9675-85b3fd34d2e8 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:10.164009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876e09e-e7d3-4b17-9675-85b3fd34d2e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:10.165375 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876e09e-e7d3-4b17-9675-85b3fd34d2e8 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:10.172468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 642/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 18:47:10 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:47:12.187687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487f888-596e-48fe-8abd-bbfbd7b0607b None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:12.188219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487f888-596e-48fe-8abd-bbfbd7b0607b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:12.188219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487f888-596e-48fe-8abd-bbfbd7b0607b None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:12.189610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 660/1302] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:47:12.195044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c02046-f415-4e0b-88e6-51b0db4c4c84 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:12.195756 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c02046-f415-4e0b-88e6-51b0db4c4c84 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:12.195899 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c02046-f415-4e0b-88e6-51b0db4c4c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:12.216335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b7c02046-f415-4e0b-88e6-51b0db4c4c84 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:47:12.217612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b7c02046-f415-4e0b-88e6-51b0db4c4c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:47:12.299074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 643/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:47:12 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 18:47:12.324842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487f888-596e-48fe-8abd-bbfbd7b0607b None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:12.428713 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:12.436836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487f888-596e-48fe-8abd-bbfbd7b0607b None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:12.473205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc94f0-c11a-42ea-a1e1-f789228fac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:12.474016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc94f0-c11a-42ea-a1e1-f789228fac01 None admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:12.474308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc94f0-c11a-42ea-a1e1-f789228fac01 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:12.474617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc94f0-c11a-42ea-a1e1-f789228fac01 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:12.475281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc94f0-c11a-42ea-a1e1-f789228fac01 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:12.476642 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc94f0-c11a-42ea-a1e1-f789228fac01 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application [None req-96fc94f0-c11a-42ea-a1e1-f789228fac01 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 18:47:12.489494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 18:47:12.491445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application with Lock( Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self._enter() Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application return self.creator() Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application created_value = creator( Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: ERROR keystone.server.flask.application Jul 25 18:47:12.493070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 661/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 18:47:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 18:47:12.497040 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c02046-f415-4e0b-88e6-51b0db4c4c84 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:12.575294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:12.598862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c02046-f415-4e0b-88e6-51b0db4c4c84 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:12.634271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9ddc8-c60a-4433-b83a-11505d2d8755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:12.635059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9ddc8-c60a-4433-b83a-11505d2d8755 None admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:12.635347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9ddc8-c60a-4433-b83a-11505d2d8755 None admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:12.635642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9ddc8-c60a-4433-b83a-11505d2d8755 None admin] PATH_INFO: `/v3/services` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:12.636353 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9ddc8-c60a-4433-b83a-11505d2d8755 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 18:47:12.636666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9ddc8-c60a-4433-b83a-11505d2d8755 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:12.638045 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9ddc8-c60a-4433-b83a-11505d2d8755 None admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:12.643470 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 644/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Jul 25 18:47:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 18:47:12.650188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc94f0-c11a-42ea-a1e1-f789228fac01 None admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:47:12.756258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc94f0-c11a-42ea-a1e1-f789228fac01 None admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:47:12.792104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb74b2-3a1d-47ce-8039-f16a4a6fd4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:47:12.792778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb74b2-3a1d-47ce-8039-f16a4a6fd4a3 None admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:47:12.793070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb74b2-3a1d-47ce-8039-f16a4a6fd4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:47:12.793351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb74b2-3a1d-47ce-8039-f16a4a6fd4a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:47:12.794105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb74b2-3a1d-47ce-8039-f16a4a6fd4a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:47:12.795594 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb74b2-3a1d-47ce-8039-f16a4a6fd4a3 None admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:47:12.897592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 662/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 18:47:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 18:47:22.702550 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=30 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:47:22.892401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:58.938958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7f67b-a0c9-41d7-bce6-85b5db8c02cf None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:58.940088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7f67b-a0c9-41d7-bce6-85b5db8c02cf None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:58.940177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7f67b-a0c9-41d7-bce6-85b5db8c02cf None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:58.947010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f9515-d39d-41fc-a17f-7f7cd8f390c8 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:58.947386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f9515-d39d-41fc-a17f-7f7cd8f390c8 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:58.947479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f9515-d39d-41fc-a17f-7f7cd8f390c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:58.952456 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 645/1307] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:54:58 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:59.031690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-fb5f9515-d39d-41fc-a17f-7f7cd8f390c8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:59.032905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fb5f9515-d39d-41fc-a17f-7f7cd8f390c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:59.174790 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 663/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:54:59.210007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47b827-9a3f-4240-ae02-a29945f31e3f None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:59.210473 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47b827-9a3f-4240-ae02-a29945f31e3f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:59.210722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47b827-9a3f-4240-ae02-a29945f31e3f None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:59.212357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 646/1309] 10.222.0.17 () {54 vars in 965 bytes} [Mon Jul 25 18:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:54:59.218908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14ea7b-c292-4633-a7bf-08cb17caf728 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:59.219170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14ea7b-c292-4633-a7bf-08cb17caf728 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:59.219436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14ea7b-c292-4633-a7bf-08cb17caf728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:59.315658 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-fc14ea7b-c292-4633-a7bf-08cb17caf728 None None] MFA Rules not processed for user `ffcf280bc6d54b2fb2d22634468ac6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:54:59.317345 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fc14ea7b-c292-4633-a7bf-08cb17caf728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:59.346584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:59.439104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 664/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Mon Jul 25 18:54:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:54:59.449755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47b827-9a3f-4240-ae02-a29945f31e3f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:54:59.451694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-cd47b827-9a3f-4240-ae02-a29945f31e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:59.608595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47b827-9a3f-4240-ae02-a29945f31e3f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:54:59.718285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb596bbe-cce0-494c-b31a-973b6b3a6ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:54:59.719154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb596bbe-cce0-494c-b31a-973b6b3a6ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:54:59.719607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb596bbe-cce0-494c-b31a-973b6b3a6ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:54:59.719923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb596bbe-cce0-494c-b31a-973b6b3a6ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:54:59.722897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bb596bbe-cce0-494c-b31a-973b6b3a6ca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:54:59.803713 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:54:59.900544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb596bbe-cce0-494c-b31a-973b6b3a6ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:54:59.903545 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb596bbe-cce0-494c-b31a-973b6b3a6ca8 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:54:59.938842 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 647/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:54:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:02.336039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4369cb9-b496-45e2-ad53-3482b3ca42dc None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:02.336561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4369cb9-b496-45e2-ad53-3482b3ca42dc None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:02.336687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4369cb9-b496-45e2-ad53-3482b3ca42dc None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:02.338203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 665/1312] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:02.343543 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff21a9-ba60-4d66-94fc-bd1ac246b82d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:02.344083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff21a9-ba60-4d66-94fc-bd1ac246b82d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:02.344164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff21a9-ba60-4d66-94fc-bd1ac246b82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:02.445533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ddff21a9-ba60-4d66-94fc-bd1ac246b82d None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:02.446837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ddff21a9-ba60-4d66-94fc-bd1ac246b82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:02.567672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 648/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:02.607017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4369cb9-b496-45e2-ad53-3482b3ca42dc None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:02.608569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d4369cb9-b496-45e2-ad53-3482b3ca42dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:02.729616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4369cb9-b496-45e2-ad53-3482b3ca42dc None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:02.819590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:02.822892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad218b81-ba0a-43fb-ad1f-ff305481a94c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:02.823623 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad218b81-ba0a-43fb-ad1f-ff305481a94c service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:02.823851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad218b81-ba0a-43fb-ad1f-ff305481a94c service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:02.824277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad218b81-ba0a-43fb-ad1f-ff305481a94c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:02.825994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ad218b81-ba0a-43fb-ad1f-ff305481a94c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:02.957445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad218b81-ba0a-43fb-ad1f-ff305481a94c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:02.958961 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad218b81-ba0a-43fb-ad1f-ff305481a94c service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:02.978925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 666/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 18:55:05.732229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0746da0-851d-4d69-9110-bcec1d8e848a None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:05.732840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0746da0-851d-4d69-9110-bcec1d8e848a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:05.732840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0746da0-851d-4d69-9110-bcec1d8e848a None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:05.734284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 649/1315] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:05.740650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5b428-51aa-4a7c-b178-0a3a5f7851ae None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:05.741197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5b428-51aa-4a7c-b178-0a3a5f7851ae None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:05.741197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5b428-51aa-4a7c-b178-0a3a5f7851ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:05.809789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-a0c5b428-51aa-4a7c-b178-0a3a5f7851ae None None] MFA Rules not processed for user `04ba33eeeb194cc6b55f828248627dbb`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:05.811070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a0c5b428-51aa-4a7c-b178-0a3a5f7851ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:05.865752 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:05.912889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 667/1316] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:05 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:05.953326 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0746da0-851d-4d69-9110-bcec1d8e848a None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:05.955766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d0746da0-851d-4d69-9110-bcec1d8e848a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:05.965848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:06.061767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0746da0-851d-4d69-9110-bcec1d8e848a None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:06.140602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac747c-a2d5-4ca5-8d18-0e1a54c5880c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:06.141256 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac747c-a2d5-4ca5-8d18-0e1a54c5880c service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:06.141539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac747c-a2d5-4ca5-8d18-0e1a54c5880c service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:06.141828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac747c-a2d5-4ca5-8d18-0e1a54c5880c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:06.143713 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-42ac747c-a2d5-4ca5-8d18-0e1a54c5880c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:06.211266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:06.260972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac747c-a2d5-4ca5-8d18-0e1a54c5880c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:06.262480 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac747c-a2d5-4ca5-8d18-0e1a54c5880c service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:06.281865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 650/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:05 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:09.245836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2e149-743d-4698-b824-791dc7ec70d9 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:09.246445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2e149-743d-4698-b824-791dc7ec70d9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:09.246445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2e149-743d-4698-b824-791dc7ec70d9 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:09.247928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 668/1318] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:09.254619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c817f3-1fe9-48ac-9104-7f0eceaea5d5 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:09.255109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c817f3-1fe9-48ac-9104-7f0eceaea5d5 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:09.255203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c817f3-1fe9-48ac-9104-7f0eceaea5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:09.323749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e0c817f3-1fe9-48ac-9104-7f0eceaea5d5 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:09.325008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e0c817f3-1fe9-48ac-9104-7f0eceaea5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:09.424399 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 651/1319] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:09.463730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2e149-743d-4698-b824-791dc7ec70d9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:09.465294 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-20b2e149-743d-4698-b824-791dc7ec70d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:09.569374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2e149-743d-4698-b824-791dc7ec70d9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:09.647215 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207758b5-25e6-4a1c-816e-e8532540b6ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:09.647887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207758b5-25e6-4a1c-816e-e8532540b6ac service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:09.648178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207758b5-25e6-4a1c-816e-e8532540b6ac service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:09.648462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207758b5-25e6-4a1c-816e-e8532540b6ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:09.650166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-207758b5-25e6-4a1c-816e-e8532540b6ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:09.658927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:09.766126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207758b5-25e6-4a1c-816e-e8532540b6ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:09.768031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207758b5-25e6-4a1c-816e-e8532540b6ac service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:09.799988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 669/1320] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:12.953992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a1dfb-8c68-49c6-8ede-f8c33b2cea3f None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:12.954774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a1dfb-8c68-49c6-8ede-f8c33b2cea3f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:12.954774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a1dfb-8c68-49c6-8ede-f8c33b2cea3f None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:12.956378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 652/1321] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:55:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:12.961864 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5b3ec-a809-401d-8757-6e4dd69cdea4 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:12.962375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5b3ec-a809-401d-8757-6e4dd69cdea4 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:12.962529 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5b3ec-a809-401d-8757-6e4dd69cdea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:13.035066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-13b5b3ec-a809-401d-8757-6e4dd69cdea4 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:13.036300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-13b5b3ec-a809-401d-8757-6e4dd69cdea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:13.100932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:13.142995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 670/1322] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:13.181526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a1dfb-8c68-49c6-8ede-f8c33b2cea3f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:13.183182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-105a1dfb-8c68-49c6-8ede-f8c33b2cea3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:13.203860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:13.297539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a1dfb-8c68-49c6-8ede-f8c33b2cea3f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:13.380226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6be882-a95c-42f1-89ae-6005d23be3d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:13.380816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6be882-a95c-42f1-89ae-6005d23be3d8 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:13.381111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6be882-a95c-42f1-89ae-6005d23be3d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:13.381394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6be882-a95c-42f1-89ae-6005d23be3d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:13.383232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0c6be882-a95c-42f1-89ae-6005d23be3d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:13.474761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:13.546653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6be882-a95c-42f1-89ae-6005d23be3d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:13.549310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6be882-a95c-42f1-89ae-6005d23be3d8 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:13.578154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 653/1323] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:15.734993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1fee3-ec52-4dd6-ac75-62456b5964e6 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:15.736426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1fee3-ec52-4dd6-ac75-62456b5964e6 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:15.736426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1fee3-ec52-4dd6-ac75-62456b5964e6 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:15.738028 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 671/1324] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:55:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:15.743688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d8249-412a-4e22-9503-e9ed134cca96 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:15.744290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d8249-412a-4e22-9503-e9ed134cca96 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:15.744290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d8249-412a-4e22-9503-e9ed134cca96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:15.819007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-011d8249-412a-4e22-9503-e9ed134cca96 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:15.820236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-011d8249-412a-4e22-9503-e9ed134cca96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:15.935950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 654/1325] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:15.990057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bad5c6-1625-4764-8099-93f4e7bb779f None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:15.990252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bad5c6-1625-4764-8099-93f4e7bb779f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:15.990570 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bad5c6-1625-4764-8099-93f4e7bb779f None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:15.991994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 672/1326] 10.222.0.17 () {54 vars in 965 bytes} [Mon Jul 25 18:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:15.998439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1da311-ad78-4a21-8915-9b9d72bd6fcd None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:15.998749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1da311-ad78-4a21-8915-9b9d72bd6fcd None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:15.999020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1da311-ad78-4a21-8915-9b9d72bd6fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:16.034281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:16.083897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3a1da311-ad78-4a21-8915-9b9d72bd6fcd None None] MFA Rules not processed for user `ffcf280bc6d54b2fb2d22634468ac6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:16.085679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3a1da311-ad78-4a21-8915-9b9d72bd6fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:16.176886 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 655/1327] 10.222.0.17 () {58 vars in 1062 bytes} [Mon Jul 25 18:55:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:16.186475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bad5c6-1625-4764-8099-93f4e7bb779f None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:16.188257 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-25bad5c6-1625-4764-8099-93f4e7bb779f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:16.301198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bad5c6-1625-4764-8099-93f4e7bb779f None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:16.381854 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692620b-9ff2-47a7-a8db-54a578d8c130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:16.382558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692620b-9ff2-47a7-a8db-54a578d8c130 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:16.382870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692620b-9ff2-47a7-a8db-54a578d8c130 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:16.383150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692620b-9ff2-47a7-a8db-54a578d8c130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:16.384817 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6692620b-9ff2-47a7-a8db-54a578d8c130 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:16.404048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:16.509605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692620b-9ff2-47a7-a8db-54a578d8c130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:16.511077 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692620b-9ff2-47a7-a8db-54a578d8c130 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:16.531357 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 673/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:18.905243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1ddac-5cb9-4efa-a9c3-b5d5c905ccbf None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:18.905985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1ddac-5cb9-4efa-a9c3-b5d5c905ccbf None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:18.905985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1ddac-5cb9-4efa-a9c3-b5d5c905ccbf None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:18.907476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 656/1329] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:18.914474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4c343-2f84-4ba6-b24d-4273e5535e47 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:18.915119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4c343-2f84-4ba6-b24d-4273e5535e47 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:18.915219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4c343-2f84-4ba6-b24d-4273e5535e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:18.992051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-b4c4c343-2f84-4ba6-b24d-4273e5535e47 None None] MFA Rules not processed for user `04ba33eeeb194cc6b55f828248627dbb`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:18.993287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b4c4c343-2f84-4ba6-b24d-4273e5535e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:19.062691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:19.096617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 674/1330] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:19.138704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c1ddac-5cb9-4efa-a9c3-b5d5c905ccbf None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:19.140713 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-31c1ddac-5cb9-4efa-a9c3-b5d5c905ccbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:19.255023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c1ddac-5cb9-4efa-a9c3-b5d5c905ccbf None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:19.276029 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:19.348164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7898f9-2598-4f23-b79f-5388b367f0e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:19.351103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7898f9-2598-4f23-b79f-5388b367f0e0 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:19.351103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7898f9-2598-4f23-b79f-5388b367f0e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:19.351103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7898f9-2598-4f23-b79f-5388b367f0e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:19.354590 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6a7898f9-2598-4f23-b79f-5388b367f0e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:19.491897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7898f9-2598-4f23-b79f-5388b367f0e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:19.493347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7898f9-2598-4f23-b79f-5388b367f0e0 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:19.512648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 657/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:19 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:22.296795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81d4c5-982b-4e9d-ab5a-b897899bfc29 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:22.297416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81d4c5-982b-4e9d-ab5a-b897899bfc29 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:22.297416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81d4c5-982b-4e9d-ab5a-b897899bfc29 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:22.298998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 675/1332] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:22.304640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61189729-ef46-4278-8fe4-a2a7d9715ef6 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:22.305137 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61189729-ef46-4278-8fe4-a2a7d9715ef6 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:22.305250 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61189729-ef46-4278-8fe4-a2a7d9715ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:22.374894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:22.378351 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-61189729-ef46-4278-8fe4-a2a7d9715ef6 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:22.380145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-61189729-ef46-4278-8fe4-a2a7d9715ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:22.488994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 658/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:22.527011 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81d4c5-982b-4e9d-ab5a-b897899bfc29 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:22.528577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8c81d4c5-982b-4e9d-ab5a-b897899bfc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:22.642500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81d4c5-982b-4e9d-ab5a-b897899bfc29 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:22.716839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b408d-43f0-4154-bbfd-b2c19e7f0702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:22.717491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b408d-43f0-4154-bbfd-b2c19e7f0702 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:22.717782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b408d-43f0-4154-bbfd-b2c19e7f0702 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:22.718065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b408d-43f0-4154-bbfd-b2c19e7f0702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:22.719931 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b38b408d-43f0-4154-bbfd-b2c19e7f0702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:22.750212 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:22.834679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b408d-43f0-4154-bbfd-b2c19e7f0702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:22.836133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b408d-43f0-4154-bbfd-b2c19e7f0702 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:22.856418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 676/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:25.261074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183ed14-575d-4c7b-8dc6-0af697f22d67 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:25.261647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183ed14-575d-4c7b-8dc6-0af697f22d67 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:25.261864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183ed14-575d-4c7b-8dc6-0af697f22d67 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:25.263325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 659/1335] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:25.271734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ffa8b-ee58-4b29-9ee9-b9c17816e9ce None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:25.272230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ffa8b-ee58-4b29-9ee9-b9c17816e9ce None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:25.272348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ffa8b-ee58-4b29-9ee9-b9c17816e9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:25.358135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-593ffa8b-ee58-4b29-9ee9-b9c17816e9ce None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:25.359392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-593ffa8b-ee58-4b29-9ee9-b9c17816e9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:25.442578 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:25.468876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 677/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:25.510523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183ed14-575d-4c7b-8dc6-0af697f22d67 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:25.512092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9183ed14-575d-4c7b-8dc6-0af697f22d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:25.623906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183ed14-575d-4c7b-8dc6-0af697f22d67 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:25.654012 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:25.705524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb768e-8220-44cd-a105-d5946b9996da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:25.706096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb768e-8220-44cd-a105-d5946b9996da service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:25.706421 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb768e-8220-44cd-a105-d5946b9996da service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:25.706704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb768e-8220-44cd-a105-d5946b9996da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:25.708404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-70eb768e-8220-44cd-a105-d5946b9996da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:25.825119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb768e-8220-44cd-a105-d5946b9996da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:25.826668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb768e-8220-44cd-a105-d5946b9996da service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:25.845785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 660/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:28.534465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b84de6-166e-4dd9-ac3c-19799adb3c05 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:28.535072 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b84de6-166e-4dd9-ac3c-19799adb3c05 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:28.535072 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b84de6-166e-4dd9-ac3c-19799adb3c05 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:28.536437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 678/1338] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:28.542058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f0fc8-fb15-4361-9ea9-1a7af50dd785 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:28.542569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f0fc8-fb15-4361-9ea9-1a7af50dd785 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:28.542707 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f0fc8-fb15-4361-9ea9-1a7af50dd785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:28.612281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-782f0fc8-fb15-4361-9ea9-1a7af50dd785 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:28.613516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-782f0fc8-fb15-4361-9ea9-1a7af50dd785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:28.622552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:28.718227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 661/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:28.756007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b84de6-166e-4dd9-ac3c-19799adb3c05 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:28.757567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-18b84de6-166e-4dd9-ac3c-19799adb3c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:28.865228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b84de6-166e-4dd9-ac3c-19799adb3c05 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:28.943224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2b755-9f43-4309-999a-2dd5908acfaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:28.943915 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2b755-9f43-4309-999a-2dd5908acfaa service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:28.944201 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2b755-9f43-4309-999a-2dd5908acfaa service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:28.944482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2b755-9f43-4309-999a-2dd5908acfaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:28.946136 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fcb2b755-9f43-4309-999a-2dd5908acfaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:28.981699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:29.057753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2b755-9f43-4309-999a-2dd5908acfaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:29.059305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2b755-9f43-4309-999a-2dd5908acfaa service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:29.079543 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 679/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:33.899554 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf1c92-961a-4551-a8f8-24645edb22e6 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:33.900293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf1c92-961a-4551-a8f8-24645edb22e6 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:33.900293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf1c92-961a-4551-a8f8-24645edb22e6 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:33.901817 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 662/1341] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:55:33.908066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b40091-d06c-4d5f-809d-2641f5208cb6 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:33.908604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b40091-d06c-4d5f-809d-2641f5208cb6 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:33.908604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b40091-d06c-4d5f-809d-2641f5208cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:33.986762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-50b40091-d06c-4d5f-809d-2641f5208cb6 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:33.988007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-50b40091-d06c-4d5f-809d-2641f5208cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:34.096353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:34.114563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 680/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:34.155931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf1c92-961a-4551-a8f8-24645edb22e6 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:34.157855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6fcf1c92-961a-4551-a8f8-24645edb22e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:34.278386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf1c92-961a-4551-a8f8-24645edb22e6 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:34.319587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:34.369398 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a129768-37e5-4e23-be19-1d03c831dc51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:34.370054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a129768-37e5-4e23-be19-1d03c831dc51 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:34.370362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a129768-37e5-4e23-be19-1d03c831dc51 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:34.370640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a129768-37e5-4e23-be19-1d03c831dc51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:34.372294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1a129768-37e5-4e23-be19-1d03c831dc51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:34.497571 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a129768-37e5-4e23-be19-1d03c831dc51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:34.499050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a129768-37e5-4e23-be19-1d03c831dc51 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:34.639918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 663/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:37.855395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1a770-4df1-4414-8ffe-dab11eebfd02 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:37.855932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1a770-4df1-4414-8ffe-dab11eebfd02 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:37.855932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1a770-4df1-4414-8ffe-dab11eebfd02 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:37.857270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 681/1344] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:37.863438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d88390-5bd8-4142-b0b8-84104cea1888 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:37.863902 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d88390-5bd8-4142-b0b8-84104cea1888 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:37.864028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d88390-5bd8-4142-b0b8-84104cea1888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:37.938543 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-11d88390-5bd8-4142-b0b8-84104cea1888 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:37.939787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-11d88390-5bd8-4142-b0b8-84104cea1888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:37.959533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:38.044950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 664/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:38.087976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1a770-4df1-4414-8ffe-dab11eebfd02 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:38.089581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-17f1a770-4df1-4414-8ffe-dab11eebfd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:38.214796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1a770-4df1-4414-8ffe-dab11eebfd02 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:38.297449 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dee06-14d6-46d8-8012-5bd40f3e5658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:38.298500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dee06-14d6-46d8-8012-5bd40f3e5658 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:38.298788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dee06-14d6-46d8-8012-5bd40f3e5658 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:38.299061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dee06-14d6-46d8-8012-5bd40f3e5658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:38.300717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-885dee06-14d6-46d8-8012-5bd40f3e5658 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:38.348033 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:38.426905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dee06-14d6-46d8-8012-5bd40f3e5658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:38.428380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dee06-14d6-46d8-8012-5bd40f3e5658 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:38.450449 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 682/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:42.887394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755584b-5607-4666-a792-5df9472e6fdf None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:42.887944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755584b-5607-4666-a792-5df9472e6fdf None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:42.887944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755584b-5607-4666-a792-5df9472e6fdf None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:42.893668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 665/1347] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:55:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:42.895439 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef1089-a79e-4014-b017-eb51b4232058 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:42.895893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef1089-a79e-4014-b017-eb51b4232058 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:42.896059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef1089-a79e-4014-b017-eb51b4232058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:42.978613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-40ef1089-a79e-4014-b017-eb51b4232058 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:42.979583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-40ef1089-a79e-4014-b017-eb51b4232058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:43.102147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:43.105317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 683/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:43.145283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755584b-5607-4666-a792-5df9472e6fdf None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:43.146891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6755584b-5607-4666-a792-5df9472e6fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:43.282766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755584b-5607-4666-a792-5df9472e6fdf None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:43.338561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:43.367922 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f9f61-3655-43e4-8eab-35742272f159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:43.368591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f9f61-3655-43e4-8eab-35742272f159 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:43.368880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f9f61-3655-43e4-8eab-35742272f159 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:43.369161 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f9f61-3655-43e4-8eab-35742272f159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:43.370968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ec4f9f61-3655-43e4-8eab-35742272f159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:43.507487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f9f61-3655-43e4-8eab-35742272f159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:55:43.508948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f9f61-3655-43e4-8eab-35742272f159 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:55:43.538629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 666/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:55:48.963843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ad178-d4c4-4a2b-8af8-e2507cec89df None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:48.964727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ad178-d4c4-4a2b-8af8-e2507cec89df None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:48.965169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ad178-d4c4-4a2b-8af8-e2507cec89df None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:48.967293 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 684/1350] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 18:55:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:48.974466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b80b4-07b9-43f8-9e8a-3d4da46dd805 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:48.974944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b80b4-07b9-43f8-9e8a-3d4da46dd805 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:48.975256 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b80b4-07b9-43f8-9e8a-3d4da46dd805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:49.071198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-191b80b4-07b9-43f8-9e8a-3d4da46dd805 None None] MFA Rules not processed for user `f5027a60da214afda9144b5ae6cce15b`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:49.072662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-191b80b4-07b9-43f8-9e8a-3d4da46dd805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:49.097920 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:49.228298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 667/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 18:55:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:53.756599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278d7fb-16cc-4388-b31b-ae7b7c2bac11 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:53.757130 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278d7fb-16cc-4388-b31b-ae7b7c2bac11 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:53.757130 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278d7fb-16cc-4388-b31b-ae7b7c2bac11 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:53.758699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 685/1352] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 18:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:53.764042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632d43-db97-47dc-a835-d96bc547da6b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:53.765219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632d43-db97-47dc-a835-d96bc547da6b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:53.765935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632d43-db97-47dc-a835-d96bc547da6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:53.896597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-a3632d43-db97-47dc-a835-d96bc547da6b None None] MFA Rules not processed for user `f5027a60da214afda9144b5ae6cce15b`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:53.898139 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a3632d43-db97-47dc-a835-d96bc547da6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:53.991235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:54.038829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 668/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 18:55:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:55.208936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3202c-1c5e-4770-b2e1-fb06719dccfa None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:55.209487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3202c-1c5e-4770-b2e1-fb06719dccfa None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:55.209487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3202c-1c5e-4770-b2e1-fb06719dccfa None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:55.211137 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 686/1354] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 18:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:55.216369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b55a5-2679-4f22-8a98-adf065ffdbaf None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:55.217021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b55a5-2679-4f22-8a98-adf065ffdbaf None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:55.217021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b55a5-2679-4f22-8a98-adf065ffdbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:55.303621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-369b55a5-2679-4f22-8a98-adf065ffdbaf None None] MFA Rules not processed for user `f5027a60da214afda9144b5ae6cce15b`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:55.304940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-369b55a5-2679-4f22-8a98-adf065ffdbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:55.423102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 669/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 18:55:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:58.992838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef7b6b-d689-4a9b-a0e0-74ccf4ce5f50 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:58.993364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef7b6b-d689-4a9b-a0e0-74ccf4ce5f50 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:58.993456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef7b6b-d689-4a9b-a0e0-74ccf4ce5f50 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:58.995232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 687/1356] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:59.004390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac6b8a-c89e-499a-b469-aa28a21e3b92 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:59.004931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac6b8a-c89e-499a-b469-aa28a21e3b92 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:59.004931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac6b8a-c89e-499a-b469-aa28a21e3b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:59.054194 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:59.104365 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-11ac6b8a-c89e-499a-b469-aa28a21e3b92 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:59.105505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-11ac6b8a-c89e-499a-b469-aa28a21e3b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:59.235409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 670/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:59.308189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d470a-9ff0-4fdf-83db-612735a29869 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:59.309056 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d470a-9ff0-4fdf-83db-612735a29869 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:59.309633 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d470a-9ff0-4fdf-83db-612735a29869 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:59.313781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 688/1358] 10.222.0.17 () {54 vars in 962 bytes} [Mon Jul 25 18:55:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:55:59.329683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574356a-fd81-42fd-8861-46d3d454b05d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:59.330082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574356a-fd81-42fd-8861-46d3d454b05d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:59.330404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574356a-fd81-42fd-8861-46d3d454b05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:59.446169 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c574356a-fd81-42fd-8861-46d3d454b05d None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:55:59.447482 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c574356a-fd81-42fd-8861-46d3d454b05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:59.503666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:59.574737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 671/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Jul 25 18:55:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:55:59.583690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d470a-9ff0-4fdf-83db-612735a29869 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:55:59.585238 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-061d470a-9ff0-4fdf-83db-612735a29869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:55:59.771483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d470a-9ff0-4fdf-83db-612735a29869 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:55:59.860241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:55:59.914391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ed7ae-e484-496a-bd3f-a7014c3a45ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:55:59.915078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ed7ae-e484-496a-bd3f-a7014c3a45ee service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:55:59.915363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ed7ae-e484-496a-bd3f-a7014c3a45ee service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:55:59.915641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ed7ae-e484-496a-bd3f-a7014c3a45ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:55:59.917978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-4f7ed7ae-e484-496a-bd3f-a7014c3a45ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:00.030768 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ed7ae-e484-496a-bd3f-a7014c3a45ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:00.033182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ed7ae-e484-496a-bd3f-a7014c3a45ee service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:00.060158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 689/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:02.359991 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ed4da-7bdf-4525-9001-eb94f3836821 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:02.360371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ed4da-7bdf-4525-9001-eb94f3836821 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:02.360646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ed4da-7bdf-4525-9001-eb94f3836821 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:02.362006 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 672/1361] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:56:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:02.367421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68c44a-9077-4561-ab7b-3f938a0b8934 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:02.367882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68c44a-9077-4561-ab7b-3f938a0b8934 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:02.367882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68c44a-9077-4561-ab7b-3f938a0b8934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:02.446303 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-5f68c44a-9077-4561-ab7b-3f938a0b8934 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:02.448740 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5f68c44a-9077-4561-ab7b-3f938a0b8934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:02.489537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:02.560994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 690/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:56:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:02.606901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ed4da-7bdf-4525-9001-eb94f3836821 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:02.608998 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-427ed4da-7bdf-4525-9001-eb94f3836821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:02.790411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ed4da-7bdf-4525-9001-eb94f3836821 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:02.803010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:02.965794 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff86299-06d8-4a7d-bd51-8350cbe75282 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:02.966567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff86299-06d8-4a7d-bd51-8350cbe75282 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:02.966972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff86299-06d8-4a7d-bd51-8350cbe75282 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:02.968028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff86299-06d8-4a7d-bd51-8350cbe75282 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:02.970338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-dff86299-06d8-4a7d-bd51-8350cbe75282 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:03.084817 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff86299-06d8-4a7d-bd51-8350cbe75282 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:03.086468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff86299-06d8-4a7d-bd51-8350cbe75282 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:03.107686 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 673/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:05.643720 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c5a30-21b6-4055-9e56-c7c685601a8b None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:05.644250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c5a30-21b6-4055-9e56-c7c685601a8b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:05.644250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c5a30-21b6-4055-9e56-c7c685601a8b None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:05.645613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 691/1364] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:05.652238 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93852118-09ef-4dad-bae6-84b124d365ad None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:05.653018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93852118-09ef-4dad-bae6-84b124d365ad None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:05.653018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93852118-09ef-4dad-bae6-84b124d365ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:05.674018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:05.743806 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-93852118-09ef-4dad-bae6-84b124d365ad None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:05.745113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-93852118-09ef-4dad-bae6-84b124d365ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:05.870380 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 674/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:05.922531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd727841-e45d-4425-bb20-9bfafb39fdfa None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:05.923105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd727841-e45d-4425-bb20-9bfafb39fdfa None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:05.923389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd727841-e45d-4425-bb20-9bfafb39fdfa None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:05.925206 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 692/1366] 10.222.0.17 () {54 vars in 962 bytes} [Mon Jul 25 18:56:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:05.930540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecb299-ed1b-4a7e-8c12-e65e4189954f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:05.936913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecb299-ed1b-4a7e-8c12-e65e4189954f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:05.936913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecb299-ed1b-4a7e-8c12-e65e4189954f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:06.031835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-aeecb299-ed1b-4a7e-8c12-e65e4189954f None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:06.033249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-aeecb299-ed1b-4a7e-8c12-e65e4189954f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:06.086710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:06.209884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 675/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Jul 25 18:56:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:06.219324 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd727841-e45d-4425-bb20-9bfafb39fdfa None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:06.220998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-cd727841-e45d-4425-bb20-9bfafb39fdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:06.393592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd727841-e45d-4425-bb20-9bfafb39fdfa None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:06.417991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:06.532877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a8f00-e96c-437f-a5c0-53dad76a238f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:06.533551 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a8f00-e96c-437f-a5c0-53dad76a238f service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:06.533844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a8f00-e96c-437f-a5c0-53dad76a238f service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:06.534131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a8f00-e96c-437f-a5c0-53dad76a238f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:06.536021 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-601a8f00-e96c-437f-a5c0-53dad76a238f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:06.649190 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a8f00-e96c-437f-a5c0-53dad76a238f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:06.651159 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a8f00-e96c-437f-a5c0-53dad76a238f service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:06.676359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 693/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:07.644925 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88be7c2-43aa-4a07-bb85-9f05e13e0426 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:07.645503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88be7c2-43aa-4a07-bb85-9f05e13e0426 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:07.645503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88be7c2-43aa-4a07-bb85-9f05e13e0426 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:07.647102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 676/1369] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:07.652753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40733095-c819-42df-bd1f-f86e0ad515b3 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:07.653231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40733095-c819-42df-bd1f-f86e0ad515b3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:07.653326 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40733095-c819-42df-bd1f-f86e0ad515b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:07.687712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:07.727249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-40733095-c819-42df-bd1f-f86e0ad515b3 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:07.728707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-40733095-c819-42df-bd1f-f86e0ad515b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:07.839719 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 694/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:08.236575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b246330-afb9-4858-b74f-2a9d1561de3a None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:08.237021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b246330-afb9-4858-b74f-2a9d1561de3a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:08.237404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b246330-afb9-4858-b74f-2a9d1561de3a None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:08.239160 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 677/1371] 10.222.0.17 () {54 vars in 964 bytes} [Mon Jul 25 18:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:08.244377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad0628-4b7f-421f-bac4-a9db6941888a None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:08.244805 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad0628-4b7f-421f-bac4-a9db6941888a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:08.245185 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad0628-4b7f-421f-bac4-a9db6941888a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:08.340986 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-cbad0628-4b7f-421f-bac4-a9db6941888a None None] MFA Rules not processed for user `9a419d02a7dc46e9b3c5b3d4eb3151d6`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:08.343661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-cbad0628-4b7f-421f-bac4-a9db6941888a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:08.402396 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:08.439721 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 695/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Mon Jul 25 18:56:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:08.449555 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b246330-afb9-4858-b74f-2a9d1561de3a None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:08.451265 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0b246330-afb9-4858-b74f-2a9d1561de3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:08.571648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b246330-afb9-4858-b74f-2a9d1561de3a None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:08.627763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:08.664565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7feb56-f080-427c-9660-56b85fd314bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:08.665502 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7feb56-f080-427c-9660-56b85fd314bf service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:08.665824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7feb56-f080-427c-9660-56b85fd314bf service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:08.666552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7feb56-f080-427c-9660-56b85fd314bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:08.668328 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bf7feb56-f080-427c-9660-56b85fd314bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:08.794196 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7feb56-f080-427c-9660-56b85fd314bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:08.796342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7feb56-f080-427c-9660-56b85fd314bf service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:08.816705 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 678/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:08.991097 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c609d5-ccaa-41ac-9729-e21999e058e5 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:08.991736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c609d5-ccaa-41ac-9729-e21999e058e5 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:08.992181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c609d5-ccaa-41ac-9729-e21999e058e5 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:08.994815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 696/1374] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:56:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:09.000535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419abdb-f0d6-4b30-a38e-07578ffcfbd4 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:09.000874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419abdb-f0d6-4b30-a38e-07578ffcfbd4 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:09.001039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419abdb-f0d6-4b30-a38e-07578ffcfbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:09.072645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e419abdb-f0d6-4b30-a38e-07578ffcfbd4 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:09.075659 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e419abdb-f0d6-4b30-a38e-07578ffcfbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:09.096479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:09.191955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 679/1375] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:09.239481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c609d5-ccaa-41ac-9729-e21999e058e5 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:09.241963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-75c609d5-ccaa-41ac-9729-e21999e058e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:09.402811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c609d5-ccaa-41ac-9729-e21999e058e5 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:09.468843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:09.538727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc457d-6619-4208-8206-a99fbdcd8728 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:09.539503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc457d-6619-4208-8206-a99fbdcd8728 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:09.539696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc457d-6619-4208-8206-a99fbdcd8728 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:09.539964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc457d-6619-4208-8206-a99fbdcd8728 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:09.541602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-24cc457d-6619-4208-8206-a99fbdcd8728 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:09.636704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc457d-6619-4208-8206-a99fbdcd8728 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:09.638352 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc457d-6619-4208-8206-a99fbdcd8728 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:09.658735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 697/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:11.934011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eff0e-0408-412c-adbf-99ae0b55dfff None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:11.934933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eff0e-0408-412c-adbf-99ae0b55dfff None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:11.935222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eff0e-0408-412c-adbf-99ae0b55dfff None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:11.936763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 680/1377] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:56:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:11.942211 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556f869-5cd5-40b5-a47d-91d85a26236a None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:11.942692 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556f869-5cd5-40b5-a47d-91d85a26236a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:11.942932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556f869-5cd5-40b5-a47d-91d85a26236a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:12.017847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-0556f869-5cd5-40b5-a47d-91d85a26236a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:12.019130 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-0556f869-5cd5-40b5-a47d-91d85a26236a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:12.032641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:12.153514 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 698/1378] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:12.200825 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6eff0e-0408-412c-adbf-99ae0b55dfff None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:12.202608 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-dc6eff0e-0408-412c-adbf-99ae0b55dfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:12.359719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6eff0e-0408-412c-adbf-99ae0b55dfff None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:12.363778 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:12.493566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee3f6b-c13e-4abb-b085-02052d7e750e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:12.494251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee3f6b-c13e-4abb-b085-02052d7e750e service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:12.494634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee3f6b-c13e-4abb-b085-02052d7e750e service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:12.494864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee3f6b-c13e-4abb-b085-02052d7e750e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:12.496764 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-21ee3f6b-c13e-4abb-b085-02052d7e750e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:12.635925 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee3f6b-c13e-4abb-b085-02052d7e750e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:12.637633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee3f6b-c13e-4abb-b085-02052d7e750e service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:12.675654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 681/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:15.033513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7eb963-3331-4f25-81e5-457e80b654ea None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:15.034049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7eb963-3331-4f25-81e5-457e80b654ea None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:15.034148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7eb963-3331-4f25-81e5-457e80b654ea None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:15.035542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 699/1380] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:15.050828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c039a6-5135-4c84-ad4b-b00274b14300 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:15.050828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c039a6-5135-4c84-ad4b-b00274b14300 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:15.050828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c039a6-5135-4c84-ad4b-b00274b14300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:15.064957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:15.132222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-f8c039a6-5135-4c84-ad4b-b00274b14300 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:15.133669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f8c039a6-5135-4c84-ad4b-b00274b14300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:15.254611 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 682/1381] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:15.305237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7eb963-3331-4f25-81e5-457e80b654ea None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:15.307458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-da7eb963-3331-4f25-81e5-457e80b654ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:15.517454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:15.536203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7eb963-3331-4f25-81e5-457e80b654ea None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:15.697657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485b6e3-80e4-4578-8664-45e2d829bb8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:15.698344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485b6e3-80e4-4578-8664-45e2d829bb8b service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:15.698654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485b6e3-80e4-4578-8664-45e2d829bb8b service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:15.698932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485b6e3-80e4-4578-8664-45e2d829bb8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:15.700588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-2485b6e3-80e4-4578-8664-45e2d829bb8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:15.806958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485b6e3-80e4-4578-8664-45e2d829bb8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:15.808516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485b6e3-80e4-4578-8664-45e2d829bb8b service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:15.820807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:15.830888 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 700/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:18.176959 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37693b52-e1d0-45ae-99ff-80294716f9d7 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:18.177526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37693b52-e1d0-45ae-99ff-80294716f9d7 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:18.177526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37693b52-e1d0-45ae-99ff-80294716f9d7 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:18.178981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 683/1383] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:56:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:18.184909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64783766-7c22-45fa-a300-cff660aa6e1c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:18.186162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64783766-7c22-45fa-a300-cff660aa6e1c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:18.186489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64783766-7c22-45fa-a300-cff660aa6e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:18.262377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-64783766-7c22-45fa-a300-cff660aa6e1c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:18.263661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-64783766-7c22-45fa-a300-cff660aa6e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:18.400439 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 701/1384] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:18.447199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37693b52-e1d0-45ae-99ff-80294716f9d7 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:18.448884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-37693b52-e1d0-45ae-99ff-80294716f9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:18.562320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:18.633926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37693b52-e1d0-45ae-99ff-80294716f9d7 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:18.798918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7414c9-6212-42ba-bd17-e8c8036a657a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:18.799505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7414c9-6212-42ba-bd17-e8c8036a657a service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:18.799791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7414c9-6212-42ba-bd17-e8c8036a657a service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:18.800075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7414c9-6212-42ba-bd17-e8c8036a657a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:18.801769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4e7414c9-6212-42ba-bd17-e8c8036a657a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:18.860512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:18.909236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7414c9-6212-42ba-bd17-e8c8036a657a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:18.910920 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7414c9-6212-42ba-bd17-e8c8036a657a service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:18.932202 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 684/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 18:56:21.058300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4f2b4-e275-412e-a9cd-12c17c878696 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:21.059018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4f2b4-e275-412e-a9cd-12c17c878696 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:21.059018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4f2b4-e275-412e-a9cd-12c17c878696 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:21.060441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 702/1386] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:21.067441 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefcbbf-3f21-4395-b41d-5581436c402a None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:21.068002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefcbbf-3f21-4395-b41d-5581436c402a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:21.068002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefcbbf-3f21-4395-b41d-5581436c402a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:21.151142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-6cefcbbf-3f21-4395-b41d-5581436c402a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:21.152770 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6cefcbbf-3f21-4395-b41d-5581436c402a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:21.267640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 685/1387] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:21.315089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4f2b4-e275-412e-a9cd-12c17c878696 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:21.316764 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d1d4f2b4-e275-412e-a9cd-12c17c878696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:21.421214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:21.533307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4f2b4-e275-412e-a9cd-12c17c878696 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:21.704500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073187f-db23-4b8b-bd10-41c9dd9ac7cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:21.705560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073187f-db23-4b8b-bd10-41c9dd9ac7cd service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:21.705560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073187f-db23-4b8b-bd10-41c9dd9ac7cd service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:21.705804 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073187f-db23-4b8b-bd10-41c9dd9ac7cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:21.707525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-0073187f-db23-4b8b-bd10-41c9dd9ac7cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:21.754488 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:21.813249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073187f-db23-4b8b-bd10-41c9dd9ac7cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:21.815754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073187f-db23-4b8b-bd10-41c9dd9ac7cd service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:21.840147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 703/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:24.225544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23abfcf-b5c0-4606-868f-c415697fcad1 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:24.226183 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23abfcf-b5c0-4606-868f-c415697fcad1 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:24.226183 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23abfcf-b5c0-4606-868f-c415697fcad1 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:24.227620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 686/1389] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:24.235961 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6411a5c-e376-465b-a013-5c8a422265f5 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:24.236795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6411a5c-e376-465b-a013-5c8a422265f5 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:24.236978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6411a5c-e376-465b-a013-5c8a422265f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:24.308800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-f6411a5c-e376-465b-a013-5c8a422265f5 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:24.310149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f6411a5c-e376-465b-a013-5c8a422265f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:24.420362 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 704/1390] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:56:24.472776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23abfcf-b5c0-4606-868f-c415697fcad1 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:24.474429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d23abfcf-b5c0-4606-868f-c415697fcad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:24.507876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:24.654868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23abfcf-b5c0-4606-868f-c415697fcad1 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:24.788677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:24.803180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc0e0c-459c-4eb9-b50f-8df86d0671d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:24.803901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc0e0c-459c-4eb9-b50f-8df86d0671d2 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:24.804194 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc0e0c-459c-4eb9-b50f-8df86d0671d2 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:24.804474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc0e0c-459c-4eb9-b50f-8df86d0671d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:24.806246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-cffc0e0c-459c-4eb9-b50f-8df86d0671d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:24.927932 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc0e0c-459c-4eb9-b50f-8df86d0671d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:24.929572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc0e0c-459c-4eb9-b50f-8df86d0671d2 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:24.955675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 687/1391] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:25.065946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45d9db-d82e-4790-96c1-e947c0b47be4 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:25.066238 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45d9db-d82e-4790-96c1-e947c0b47be4 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:25.066595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45d9db-d82e-4790-96c1-e947c0b47be4 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:25.067998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 705/1392] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:25.073738 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63f655-0f7b-46d6-b65a-2c10468ab5ed None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:25.074076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63f655-0f7b-46d6-b65a-2c10468ab5ed None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:25.074361 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63f655-0f7b-46d6-b65a-2c10468ab5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:25.169044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ac63f655-0f7b-46d6-b65a-2c10468ab5ed None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:25.171230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ac63f655-0f7b-46d6-b65a-2c10468ab5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:25.257716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:25.327982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 688/1393] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:25.390700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19df33-a1b9-4e03-8741-91c69d7d71a4 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:25.391224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19df33-a1b9-4e03-8741-91c69d7d71a4 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:25.391330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19df33-a1b9-4e03-8741-91c69d7d71a4 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:25.392900 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 706/1394] 10.222.0.17 () {54 vars in 966 bytes} [Mon Jul 25 18:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:25.401701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6f5f7-07cf-4e51-914d-5e5055f3fc64 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:25.402104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6f5f7-07cf-4e51-914d-5e5055f3fc64 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:25.402397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6f5f7-07cf-4e51-914d-5e5055f3fc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:25.505754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-75f6f5f7-07cf-4e51-914d-5e5055f3fc64 None None] MFA Rules not processed for user `28954d67689a46ee975fdfc9139e41eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:25.507044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-75f6f5f7-07cf-4e51-914d-5e5055f3fc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:25.589514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 689/1395] 10.222.0.17 () {58 vars in 1063 bytes} [Mon Jul 25 18:56:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:25.597917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19df33-a1b9-4e03-8741-91c69d7d71a4 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:25.599487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3a19df33-a1b9-4e03-8741-91c69d7d71a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:25.615070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:25.726687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19df33-a1b9-4e03-8741-91c69d7d71a4 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:25.811738 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a6bfb-9d91-4fab-bb17-c522c10abe80 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28954d67689a46ee975fdfc9139e41eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:25.812603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a6bfb-9d91-4fab-bb17-c522c10abe80 service glance] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:25.812897 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a6bfb-9d91-4fab-bb17-c522c10abe80 service glance] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:25.813206 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a6bfb-9d91-4fab-bb17-c522c10abe80 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:25.818217 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e17a6bfb-9d91-4fab-bb17-c522c10abe80 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:25.920418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:25.965710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a6bfb-9d91-4fab-bb17-c522c10abe80 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:25.967785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a6bfb-9d91-4fab-bb17-c522c10abe80 service glance] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:25.987831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 707/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 18:56:26.883924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19133313-13c8-4224-9b64-9e7286627d8d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:26.884452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19133313-13c8-4224-9b64-9e7286627d8d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:26.884547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19133313-13c8-4224-9b64-9e7286627d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:26.960729 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-19133313-13c8-4224-9b64-9e7286627d8d None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:26.962020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-19133313-13c8-4224-9b64-9e7286627d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:26.973017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:27.063165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 690/1397] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 18:56:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:27.113295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944052f5-59aa-40f3-b44e-32dd96875a53 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:27.115012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944052f5-59aa-40f3-b44e-32dd96875a53 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:27.115012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944052f5-59aa-40f3-b44e-32dd96875a53 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:27.115584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 708/1398] 10.222.0.17 () {54 vars in 943 bytes} [Mon Jul 25 18:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:27.122463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12359e-c1e1-4c07-8f03-6220308d3861 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:27.123402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12359e-c1e1-4c07-8f03-6220308d3861 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:27.123685 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12359e-c1e1-4c07-8f03-6220308d3861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:27.209907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-4b12359e-c1e1-4c07-8f03-6220308d3861 None None] MFA Rules not processed for user `1ee9152572224cc499229e0fa2b81ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:27.211203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4b12359e-c1e1-4c07-8f03-6220308d3861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:27.295333 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a832f-1b01-4345-a93f-071201d256e6 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:27.295619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a832f-1b01-4345-a93f-071201d256e6 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:27.295993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a832f-1b01-4345-a93f-071201d256e6 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:27.297469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 709/1399] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:56:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 18:56:27.303364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78aa1a-9d54-496b-a98f-fd15ff938f9f None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:27.303654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78aa1a-9d54-496b-a98f-fd15ff938f9f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:27.303944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78aa1a-9d54-496b-a98f-fd15ff938f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:27.305226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:27.330119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 691/1400] 10.222.0.17 () {58 vars in 1040 bytes} [Mon Jul 25 18:56:27 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:27.339666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12359e-c1e1-4c07-8f03-6220308d3861 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:27.351789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12359e-c1e1-4c07-8f03-6220308d3861 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:27.376509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-7a78aa1a-9d54-496b-a98f-fd15ff938f9f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:27.377670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7a78aa1a-9d54-496b-a98f-fd15ff938f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:27.465133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51edf1a-77e8-43c0-9ca2-e4b2927b22e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:27.465796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51edf1a-77e8-43c0-9ca2-e4b2927b22e5 service swift] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:27.466116 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51edf1a-77e8-43c0-9ca2-e4b2927b22e5 service swift] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:27.467032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51edf1a-77e8-43c0-9ca2-e4b2927b22e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:27.468787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c51edf1a-77e8-43c0-9ca2-e4b2927b22e5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:27.484644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 710/1401] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:56:27.532635 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78aa1a-9d54-496b-a98f-fd15ff938f9f None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:27.533865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7a78aa1a-9d54-496b-a98f-fd15ff938f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:27.561041 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:27.561431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51edf1a-77e8-43c0-9ca2-e4b2927b22e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:27.563126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51edf1a-77e8-43c0-9ca2-e4b2927b22e5 service swift] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:27.571702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 692/1402] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 18:56:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 18:56:27.653092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78aa1a-9d54-496b-a98f-fd15ff938f9f None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:27.771420 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b6713-96d8-4c40-9d8d-496f37f78158 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:27.772083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b6713-96d8-4c40-9d8d-496f37f78158 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:27.772368 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b6713-96d8-4c40-9d8d-496f37f78158 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:27.772644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b6713-96d8-4c40-9d8d-496f37f78158 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:27.774601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-679b6713-96d8-4c40-9d8d-496f37f78158 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:27.793996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:27.871558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b6713-96d8-4c40-9d8d-496f37f78158 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:27.873381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b6713-96d8-4c40-9d8d-496f37f78158 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:27.892927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 711/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 18:56:30.185639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fa03c-fa09-4997-8caf-4f9333a9f56f None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:30.186258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fa03c-fa09-4997-8caf-4f9333a9f56f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:30.186580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fa03c-fa09-4997-8caf-4f9333a9f56f None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:30.188773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 693/1404] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:56:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:30.195018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6de69-a3f0-4dba-8f3a-92200d04c31b None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:30.195018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6de69-a3f0-4dba-8f3a-92200d04c31b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:30.195697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6de69-a3f0-4dba-8f3a-92200d04c31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:30.402496 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-a1c6de69-a3f0-4dba-8f3a-92200d04c31b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:30.404411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a1c6de69-a3f0-4dba-8f3a-92200d04c31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:30.531247 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:30.537344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 712/1405] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:30.584821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fa03c-fa09-4997-8caf-4f9333a9f56f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:30.589228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-cf0fa03c-fa09-4997-8caf-4f9333a9f56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:30.597972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:30.767642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fa03c-fa09-4997-8caf-4f9333a9f56f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:30.894507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:30.933042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514b76b-b5f6-49ed-af81-ce83278ce5f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:30.933851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514b76b-b5f6-49ed-af81-ce83278ce5f3 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:30.934141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514b76b-b5f6-49ed-af81-ce83278ce5f3 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:30.934500 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514b76b-b5f6-49ed-af81-ce83278ce5f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:30.936824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6514b76b-b5f6-49ed-af81-ce83278ce5f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:31.054425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514b76b-b5f6-49ed-af81-ce83278ce5f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:31.056254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514b76b-b5f6-49ed-af81-ce83278ce5f3 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:31.110678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 694/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:33.406715 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326b9f5-14d3-4a84-a72a-810a6bb9ed1c None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:33.407344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326b9f5-14d3-4a84-a72a-810a6bb9ed1c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:33.407344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326b9f5-14d3-4a84-a72a-810a6bb9ed1c None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:33.410981 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 713/1407] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:56:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:33.414420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8b6a5-c799-4dd1-8546-31f3029e55bf None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:33.414908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8b6a5-c799-4dd1-8546-31f3029e55bf None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:33.414998 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8b6a5-c799-4dd1-8546-31f3029e55bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:33.495085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-00c8b6a5-c799-4dd1-8546-31f3029e55bf None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:33.496319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-00c8b6a5-c799-4dd1-8546-31f3029e55bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:33.554688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:33.623990 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 695/1408] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:33.671967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326b9f5-14d3-4a84-a72a-810a6bb9ed1c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:33.673803 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1326b9f5-14d3-4a84-a72a-810a6bb9ed1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:33.847490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326b9f5-14d3-4a84-a72a-810a6bb9ed1c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:33.942342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:33.998921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f941eaa-ed04-4d69-8e54-0d719569760d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:33.999905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f941eaa-ed04-4d69-8e54-0d719569760d service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:34.000366 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f941eaa-ed04-4d69-8e54-0d719569760d service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:34.000646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f941eaa-ed04-4d69-8e54-0d719569760d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:34.002450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5f941eaa-ed04-4d69-8e54-0d719569760d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:34.121018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f941eaa-ed04-4d69-8e54-0d719569760d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:34.126662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f941eaa-ed04-4d69-8e54-0d719569760d service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:34.154283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 714/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 18:56:43.619089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=23 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:44.152129 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=64 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:52.048304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd829ad0-54d6-4836-b57c-d06add8049a6 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:52.048881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd829ad0-54d6-4836-b57c-d06add8049a6 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:52.048881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd829ad0-54d6-4836-b57c-d06add8049a6 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:52.050275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 696/1410] 10.222.0.17 () {54 vars in 918 bytes} [Mon Jul 25 18:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:52.055671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288533f8-38ff-4362-91f8-41bb2c77ea02 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:52.056126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288533f8-38ff-4362-91f8-41bb2c77ea02 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:52.056227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288533f8-38ff-4362-91f8-41bb2c77ea02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:52.135661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-288533f8-38ff-4362-91f8-41bb2c77ea02 None None] MFA Rules not processed for user `f5027a60da214afda9144b5ae6cce15b`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:52.136893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-288533f8-38ff-4362-91f8-41bb2c77ea02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:52.255007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 715/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Jul 25 18:56:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:52.270864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a141f-8d6a-42cd-bf77-b35d0ebc42d7 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:52.271155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a141f-8d6a-42cd-bf77-b35d0ebc42d7 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:52.271435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a141f-8d6a-42cd-bf77-b35d0ebc42d7 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:52.272874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 697/1412] 10.222.0.17 () {54 vars in 918 bytes} [Mon Jul 25 18:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:52.278419 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf97d1a-47c0-4d8f-b7cd-d68ab0edb582 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:52.279258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf97d1a-47c0-4d8f-b7cd-d68ab0edb582 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:52.279740 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf97d1a-47c0-4d8f-b7cd-d68ab0edb582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:52.360821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:52.363416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-daf97d1a-47c0-4d8f-b7cd-d68ab0edb582 None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:52.364658 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-daf97d1a-47c0-4d8f-b7cd-d68ab0edb582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:52.486920 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 716/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Jul 25 18:56:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:55.005073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07157b-c9cc-4cb4-8415-3abfcf534cbc None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:55.005701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07157b-c9cc-4cb4-8415-3abfcf534cbc None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:55.005701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07157b-c9cc-4cb4-8415-3abfcf534cbc None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:55.007039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:55.012476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1cce6-6fdc-413c-9fac-fd636c16e3a1 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:55.012902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1cce6-6fdc-413c-9fac-fd636c16e3a1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:55.013057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1cce6-6fdc-413c-9fac-fd636c16e3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:55.083042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-5fc1cce6-6fdc-413c-9fac-fd636c16e3a1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:55.084320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5fc1cce6-6fdc-413c-9fac-fd636c16e3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:55.142959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:55.188435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:55.233688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07157b-c9cc-4cb4-8415-3abfcf534cbc None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:55.235280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1e07157b-c9cc-4cb4-8415-3abfcf534cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:55.391272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07157b-c9cc-4cb4-8415-3abfcf534cbc None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:55.468989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:55.510844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1369c2a-60c8-4b50-b020-e03b81ff1e7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:55.511599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1369c2a-60c8-4b50-b020-e03b81ff1e7d service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:55.512124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1369c2a-60c8-4b50-b020-e03b81ff1e7d service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:55.512436 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1369c2a-60c8-4b50-b020-e03b81ff1e7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:55.515934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c1369c2a-60c8-4b50-b020-e03b81ff1e7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:55.615003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1369c2a-60c8-4b50-b020-e03b81ff1e7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:55.616608 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1369c2a-60c8-4b50-b020-e03b81ff1e7d service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:55.635694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 699/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:56.576777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76086f-94cd-4847-a780-65fb80f96e6e None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:56.577443 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76086f-94cd-4847-a780-65fb80f96e6e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:56.577443 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76086f-94cd-4847-a780-65fb80f96e6e None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:56.579100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:56:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:56.585289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d7160-255d-4f23-93ae-7c787be3c55b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:56.585289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d7160-255d-4f23-93ae-7c787be3c55b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:56.586617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d7160-255d-4f23-93ae-7c787be3c55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:56.668456 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-f15d7160-255d-4f23-93ae-7c787be3c55b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:56.670204 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f15d7160-255d-4f23-93ae-7c787be3c55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:56.698754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:56.769099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:56.927478 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76086f-94cd-4847-a780-65fb80f96e6e None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:56.929022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-bf76086f-94cd-4847-a780-65fb80f96e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:57.095767 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76086f-94cd-4847-a780-65fb80f96e6e None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:57.139697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:57.220538 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a687894-269a-449a-b1d5-c23186a1ac7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:57.221196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a687894-269a-449a-b1d5-c23186a1ac7d service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:57.221480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a687894-269a-449a-b1d5-c23186a1ac7d service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:57.221755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a687894-269a-449a-b1d5-c23186a1ac7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:57.223425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7a687894-269a-449a-b1d5-c23186a1ac7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:57.337952 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a687894-269a-449a-b1d5-c23186a1ac7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:57.340216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a687894-269a-449a-b1d5-c23186a1ac7d service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:57.362799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 719/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:56:58.335696 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdef8b-e8e6-49a1-bfa7-c712c182259e None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:58.336241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdef8b-e8e6-49a1-bfa7-c712c182259e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:58.336241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdef8b-e8e6-49a1-bfa7-c712c182259e None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:58.337654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:56:58.342988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0c992-1006-4370-b307-363a12e8d788 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:58.343495 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0c992-1006-4370-b307-363a12e8d788 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:58.343604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0c992-1006-4370-b307-363a12e8d788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:58.403524 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:58.425921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-22e0c992-1006-4370-b307-363a12e8d788 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:56:58.427085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-22e0c992-1006-4370-b307-363a12e8d788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:58.528294 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:56:58.699754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdef8b-e8e6-49a1-bfa7-c712c182259e None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:56:58.702662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-73bdef8b-e8e6-49a1-bfa7-c712c182259e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:58.856231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdef8b-e8e6-49a1-bfa7-c712c182259e None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:56:58.869070 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:56:58.975307 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9425176-7f04-4942-8bdd-0e70a72741de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:56:58.975903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9425176-7f04-4942-8bdd-0e70a72741de service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:56:58.976225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9425176-7f04-4942-8bdd-0e70a72741de service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:56:58.976510 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9425176-7f04-4942-8bdd-0e70a72741de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:56:58.978233 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b9425176-7f04-4942-8bdd-0e70a72741de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:56:59.071432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9425176-7f04-4942-8bdd-0e70a72741de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:56:59.073151 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9425176-7f04-4942-8bdd-0e70a72741de service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:56:59.093087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 702/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:00.067362 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b33be-6927-4779-9d04-c6eadc2a50c8 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:00.067902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b33be-6927-4779-9d04-c6eadc2a50c8 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:00.067902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b33be-6927-4779-9d04-c6eadc2a50c8 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:00.069478 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:00.074916 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec36b1a-0640-4dec-8d0b-aaead0c57dae None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:00.075913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec36b1a-0640-4dec-8d0b-aaead0c57dae None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:00.076210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec36b1a-0640-4dec-8d0b-aaead0c57dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:00.098558 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:00.158529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-4ec36b1a-0640-4dec-8d0b-aaead0c57dae None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:00.159775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4ec36b1a-0640-4dec-8d0b-aaead0c57dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:00.260369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:00.418386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b33be-6927-4779-9d04-c6eadc2a50c8 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:00.420371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a86b33be-6927-4779-9d04-c6eadc2a50c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:00.534949 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:00.573329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b33be-6927-4779-9d04-c6eadc2a50c8 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:00.697566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac4a80-d2c2-4084-86f7-14ee871e76d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:00.698229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac4a80-d2c2-4084-86f7-14ee871e76d8 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:00.698540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac4a80-d2c2-4084-86f7-14ee871e76d8 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:00.698820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac4a80-d2c2-4084-86f7-14ee871e76d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:00.701129 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-81ac4a80-d2c2-4084-86f7-14ee871e76d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:00.777877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:00.792850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac4a80-d2c2-4084-86f7-14ee871e76d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:00.801277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac4a80-d2c2-4084-86f7-14ee871e76d8 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:00.816036 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 722/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:01.720171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a14c9a-8ac0-4c41-b338-ebf3ad48679c None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:01.720806 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a14c9a-8ac0-4c41-b338-ebf3ad48679c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:01.720806 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a14c9a-8ac0-4c41-b338-ebf3ad48679c None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:01.722264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:01.727856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea6a2a-1b4f-4e86-af73-b3f9b5c4006a None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:01.728357 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea6a2a-1b4f-4e86-af73-b3f9b5c4006a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:01.728451 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea6a2a-1b4f-4e86-af73-b3f9b5c4006a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:01.823433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-aeea6a2a-1b4f-4e86-af73-b3f9b5c4006a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:01.824661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-aeea6a2a-1b4f-4e86-af73-b3f9b5c4006a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:01.925940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:02.084275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a14c9a-8ac0-4c41-b338-ebf3ad48679c None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:02.084275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-85a14c9a-8ac0-4c41-b338-ebf3ad48679c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:02.175332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:02.240354 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a14c9a-8ac0-4c41-b338-ebf3ad48679c None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:02.359664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e3554-e24b-4fff-ba23-092b05652943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:02.360234 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e3554-e24b-4fff-ba23-092b05652943 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:02.360514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e3554-e24b-4fff-ba23-092b05652943 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:02.360783 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e3554-e24b-4fff-ba23-092b05652943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:02.362477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3e7e3554-e24b-4fff-ba23-092b05652943 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:02.413553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:02.460797 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e3554-e24b-4fff-ba23-092b05652943 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:02.462415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e3554-e24b-4fff-ba23-092b05652943 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:02.482038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 705/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:03.394605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084e285-de91-44bb-bcac-ce44db264a60 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:03.395180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084e285-de91-44bb-bcac-ce44db264a60 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:03.395180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084e285-de91-44bb-bcac-ce44db264a60 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:03.396570 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:03.403003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b5222-c585-4881-9484-62e9531ec1e8 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:03.403669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b5222-c585-4881-9484-62e9531ec1e8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:03.403924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b5222-c585-4881-9484-62e9531ec1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:03.487754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-5b8b5222-c585-4881-9484-62e9531ec1e8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:03.488988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5b8b5222-c585-4881-9484-62e9531ec1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:03.593803 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:03.749923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084e285-de91-44bb-bcac-ce44db264a60 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:03.751599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a084e285-de91-44bb-bcac-ce44db264a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:03.802664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:03.904794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084e285-de91-44bb-bcac-ce44db264a60 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:04.026346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd92c9-8730-4e7c-8c20-9032df1d3901 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:04.026911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd92c9-8730-4e7c-8c20-9032df1d3901 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:04.027199 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd92c9-8730-4e7c-8c20-9032df1d3901 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:04.027481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd92c9-8730-4e7c-8c20-9032df1d3901 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:04.029213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c3bd92c9-8730-4e7c-8c20-9032df1d3901 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:04.036872 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:04.125115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd92c9-8730-4e7c-8c20-9032df1d3901 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:04.126767 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd92c9-8730-4e7c-8c20-9032df1d3901 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:04.147877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 725/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:05.037242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55e4dd-69db-43a6-97cb-274041e6da02 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:05.037713 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55e4dd-69db-43a6-97cb-274041e6da02 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:05.038190 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55e4dd-69db-43a6-97cb-274041e6da02 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:05.043769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:05.046139 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008fa25-40bb-40c0-8cef-4d34d1b128c2 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:05.046689 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008fa25-40bb-40c0-8cef-4d34d1b128c2 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:05.047200 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008fa25-40bb-40c0-8cef-4d34d1b128c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:05.134945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-f008fa25-40bb-40c0-8cef-4d34d1b128c2 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:05.136299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f008fa25-40bb-40c0-8cef-4d34d1b128c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:05.209657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:05.240629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:05.395990 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55e4dd-69db-43a6-97cb-274041e6da02 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:05.397965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5c55e4dd-69db-43a6-97cb-274041e6da02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:05.416409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:05.557186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55e4dd-69db-43a6-97cb-274041e6da02 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:05.649892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:05.677056 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c843bc-a54f-4ca1-8270-b3525a916264 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:05.677717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c843bc-a54f-4ca1-8270-b3525a916264 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:05.678006 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c843bc-a54f-4ca1-8270-b3525a916264 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:05.678293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c843bc-a54f-4ca1-8270-b3525a916264 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:05.679988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-97c843bc-a54f-4ca1-8270-b3525a916264 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:05.774075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c843bc-a54f-4ca1-8270-b3525a916264 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:05.775755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c843bc-a54f-4ca1-8270-b3525a916264 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:05.795940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 708/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:06.760071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509d19e-d810-42b7-aaf4-c8e453278762 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:06.760690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509d19e-d810-42b7-aaf4-c8e453278762 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:06.760690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509d19e-d810-42b7-aaf4-c8e453278762 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:06.762117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:57:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:06.769743 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c85616-bd7f-4ba5-a141-e55f1403c8bc None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:06.770448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c85616-bd7f-4ba5-a141-e55f1403c8bc None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:06.770448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c85616-bd7f-4ba5-a141-e55f1403c8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:06.857096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e7c85616-bd7f-4ba5-a141-e55f1403c8bc None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:06.858432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e7c85616-bd7f-4ba5-a141-e55f1403c8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:06.901539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:06.961538 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:07.120726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509d19e-d810-42b7-aaf4-c8e453278762 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:07.122817 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8509d19e-d810-42b7-aaf4-c8e453278762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:07.282686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509d19e-d810-42b7-aaf4-c8e453278762 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:07.352622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:07.422576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3f4fe-8326-4864-a666-f36317f32e40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:07.423153 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3f4fe-8326-4864-a666-f36317f32e40 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:07.423441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3f4fe-8326-4864-a666-f36317f32e40 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:07.423724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3f4fe-8326-4864-a666-f36317f32e40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:07.425437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-bfe3f4fe-8326-4864-a666-f36317f32e40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:07.538425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3f4fe-8326-4864-a666-f36317f32e40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:07.540082 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3f4fe-8326-4864-a666-f36317f32e40 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:07.559185 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 728/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:08.468555 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b9c56-8817-4eda-96ac-285a2f5da7b2 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:08.469130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b9c56-8817-4eda-96ac-285a2f5da7b2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:08.469130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b9c56-8817-4eda-96ac-285a2f5da7b2 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:08.470598 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 18:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:08.475780 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35670c43-859b-4add-a7c9-fde63d542766 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:08.476237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35670c43-859b-4add-a7c9-fde63d542766 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:08.476346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35670c43-859b-4add-a7c9-fde63d542766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:08.563546 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-35670c43-859b-4add-a7c9-fde63d542766 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:08.564879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-35670c43-859b-4add-a7c9-fde63d542766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:08.567274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:08.682401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 18:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:08.859047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b9c56-8817-4eda-96ac-285a2f5da7b2 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:08.860478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-785b9c56-8817-4eda-96ac-285a2f5da7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:09.027421 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b9c56-8817-4eda-96ac-285a2f5da7b2 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:09.057142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:09.149675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93271f-2423-4a7e-8f46-122f4fa37e35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:09.150257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93271f-2423-4a7e-8f46-122f4fa37e35 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:09.150572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93271f-2423-4a7e-8f46-122f4fa37e35 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:09.150850 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93271f-2423-4a7e-8f46-122f4fa37e35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:09.152621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8a93271f-2423-4a7e-8f46-122f4fa37e35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:09.250973 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93271f-2423-4a7e-8f46-122f4fa37e35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:09.252638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93271f-2423-4a7e-8f46-122f4fa37e35 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:09.271728 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 711/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:11.221263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394985ca-28d6-4064-b41a-a9aad1e55923 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:11.221831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394985ca-28d6-4064-b41a-a9aad1e55923 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:11.221831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394985ca-28d6-4064-b41a-a9aad1e55923 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:11.223259 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:11.228898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde45daa-114c-4be4-8edb-9ccd89d38a0f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:11.229377 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde45daa-114c-4be4-8edb-9ccd89d38a0f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:11.229578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde45daa-114c-4be4-8edb-9ccd89d38a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:11.274450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:11.303198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-bde45daa-114c-4be4-8edb-9ccd89d38a0f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:11.304681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bde45daa-114c-4be4-8edb-9ccd89d38a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:11.418624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:11.464605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394985ca-28d6-4064-b41a-a9aad1e55923 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:11.467062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-394985ca-28d6-4064-b41a-a9aad1e55923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:11.586577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394985ca-28d6-4064-b41a-a9aad1e55923 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:11.625773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:11.690811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79300e74-d2ad-4b12-bd2b-03a37652c2dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:11.690811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79300e74-d2ad-4b12-bd2b-03a37652c2dd service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:11.690811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79300e74-d2ad-4b12-bd2b-03a37652c2dd service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:11.691279 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79300e74-d2ad-4b12-bd2b-03a37652c2dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:11.697236 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-79300e74-d2ad-4b12-bd2b-03a37652c2dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:11.861368 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79300e74-d2ad-4b12-bd2b-03a37652c2dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:11.862970 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79300e74-d2ad-4b12-bd2b-03a37652c2dd service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:11.890673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 731/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:14.101652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41c93-6278-4521-b57f-438b97ba361f None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:14.102224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41c93-6278-4521-b57f-438b97ba361f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:14.102224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41c93-6278-4521-b57f-438b97ba361f None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:14.103640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:14.110021 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c41ebe-ebef-4d63-ad28-eb40f6ba461d None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:14.110577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c41ebe-ebef-4d63-ad28-eb40f6ba461d None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:14.110723 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c41ebe-ebef-4d63-ad28-eb40f6ba461d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:14.179376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-a4c41ebe-ebef-4d63-ad28-eb40f6ba461d None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:14.180657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a4c41ebe-ebef-4d63-ad28-eb40f6ba461d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:14.194381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:14.284004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:14.324427 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d41c93-6278-4521-b57f-438b97ba361f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:14.326054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-05d41c93-6278-4521-b57f-438b97ba361f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:14.447176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d41c93-6278-4521-b57f-438b97ba361f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:14.460339 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:14.525080 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950596e-5402-4d5f-9e84-639c604dcf23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:14.525737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950596e-5402-4d5f-9e84-639c604dcf23 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:14.526026 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950596e-5402-4d5f-9e84-639c604dcf23 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:14.526306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950596e-5402-4d5f-9e84-639c604dcf23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:14.528634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-7950596e-5402-4d5f-9e84-639c604dcf23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:14.640287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950596e-5402-4d5f-9e84-639c604dcf23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:14.641739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950596e-5402-4d5f-9e84-639c604dcf23 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:14.662133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 714/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:17.483519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c060201-d9d7-428f-9411-29fe230c3e5b None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:17.484108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c060201-d9d7-428f-9411-29fe230c3e5b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:17.484108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c060201-d9d7-428f-9411-29fe230c3e5b None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:17.485463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:17.491390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a335cd1-7043-4ed0-8c33-c092e0a79988 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:17.491990 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a335cd1-7043-4ed0-8c33-c092e0a79988 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:17.491990 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a335cd1-7043-4ed0-8c33-c092e0a79988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:17.545057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:17.564324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-0a335cd1-7043-4ed0-8c33-c092e0a79988 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:17.565565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0a335cd1-7043-4ed0-8c33-c092e0a79988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:17.672880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:17.712519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c060201-d9d7-428f-9411-29fe230c3e5b None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:17.714224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1c060201-d9d7-428f-9411-29fe230c3e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:17.826183 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c060201-d9d7-428f-9411-29fe230c3e5b None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:17.886523 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:17.924442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f04b0-4e17-472a-9417-2365c54c5091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:17.925113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f04b0-4e17-472a-9417-2365c54c5091 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:17.925402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f04b0-4e17-472a-9417-2365c54c5091 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:17.925771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f04b0-4e17-472a-9417-2365c54c5091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:17.927905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-813f04b0-4e17-472a-9417-2365c54c5091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:18.060831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f04b0-4e17-472a-9417-2365c54c5091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:18.062963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f04b0-4e17-472a-9417-2365c54c5091 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:18.095526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 734/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:20.570298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fef17-4976-4fb2-bb73-6bf3c3468a29 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:20.570908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fef17-4976-4fb2-bb73-6bf3c3468a29 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:20.570908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fef17-4976-4fb2-bb73-6bf3c3468a29 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:20.572291 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:20.578525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976ffd7-af06-4c8e-a284-d467ed8f5f20 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:20.579288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976ffd7-af06-4c8e-a284-d467ed8f5f20 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:20.579288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976ffd7-af06-4c8e-a284-d467ed8f5f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:20.671624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d976ffd7-af06-4c8e-a284-d467ed8f5f20 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:20.672849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d976ffd7-af06-4c8e-a284-d467ed8f5f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:20.698606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:20.788404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:20.835666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fef17-4976-4fb2-bb73-6bf3c3468a29 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:20.837382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b54fef17-4976-4fb2-bb73-6bf3c3468a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:20.997633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fef17-4976-4fb2-bb73-6bf3c3468a29 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:21.022768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:21.107202 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3a1a3-e3fa-4d4f-bedd-67f99f6487a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:21.107784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3a1a3-e3fa-4d4f-bedd-67f99f6487a7 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:21.108075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3a1a3-e3fa-4d4f-bedd-67f99f6487a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:21.108360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3a1a3-e3fa-4d4f-bedd-67f99f6487a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:21.110051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0ba3a1a3-e3fa-4d4f-bedd-67f99f6487a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:21.261872 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3a1a3-e3fa-4d4f-bedd-67f99f6487a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:21.263384 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3a1a3-e3fa-4d4f-bedd-67f99f6487a7 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:21.284332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 717/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:23.595682 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eebcb3-9379-433c-877a-5023739827a9 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:23.597119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eebcb3-9379-433c-877a-5023739827a9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:23.597119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eebcb3-9379-433c-877a-5023739827a9 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:23.597749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:23.605981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5727a-ec7d-45a1-b9a5-a25a6e7c8e3e None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:23.606657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5727a-ec7d-45a1-b9a5-a25a6e7c8e3e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:23.606657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5727a-ec7d-45a1-b9a5-a25a6e7c8e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:23.691264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:23.694749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e8b5727a-ec7d-45a1-b9a5-a25a6e7c8e3e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:23.696007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e8b5727a-ec7d-45a1-b9a5-a25a6e7c8e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:23.820214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:23.868623 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eebcb3-9379-433c-877a-5023739827a9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:23.870201 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d6eebcb3-9379-433c-877a-5023739827a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:24.018831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eebcb3-9379-433c-877a-5023739827a9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:24.076820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:24.107549 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17986c68-293f-4a75-8062-550cd3e816f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:24.110116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17986c68-293f-4a75-8062-550cd3e816f6 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:24.110517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17986c68-293f-4a75-8062-550cd3e816f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:24.110817 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17986c68-293f-4a75-8062-550cd3e816f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:24.115615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-17986c68-293f-4a75-8062-550cd3e816f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:24.269428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17986c68-293f-4a75-8062-550cd3e816f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:24.271020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17986c68-293f-4a75-8062-550cd3e816f6 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:24.294815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 737/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:26.633638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4506f0-7f10-4833-b16c-0591b9369911 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:26.634208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4506f0-7f10-4833-b16c-0591b9369911 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:26.634208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4506f0-7f10-4833-b16c-0591b9369911 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:26.635636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:26.641294 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444a50b-252d-408b-a9f6-1b1e40e34365 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:26.641765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444a50b-252d-408b-a9f6-1b1e40e34365 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:26.641929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444a50b-252d-408b-a9f6-1b1e40e34365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:26.710589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-8444a50b-252d-408b-a9f6-1b1e40e34365 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:26.711827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8444a50b-252d-408b-a9f6-1b1e40e34365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:26.742483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:26.812236 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:26.851044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4506f0-7f10-4833-b16c-0591b9369911 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:26.852743 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6a4506f0-7f10-4833-b16c-0591b9369911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:26.969639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4506f0-7f10-4833-b16c-0591b9369911 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:27.001695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:27.052179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340a53e-0022-4e76-a5b7-99dfeea041db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:27.053109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340a53e-0022-4e76-a5b7-99dfeea041db service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:27.053962 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340a53e-0022-4e76-a5b7-99dfeea041db service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:27.054478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340a53e-0022-4e76-a5b7-99dfeea041db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:27.056858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8340a53e-0022-4e76-a5b7-99dfeea041db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:27.178014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340a53e-0022-4e76-a5b7-99dfeea041db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:27.179475 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340a53e-0022-4e76-a5b7-99dfeea041db service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:27.199896 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 720/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:29.401031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee584f5-76dd-4c8a-98de-1f850b110d44 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:29.401592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee584f5-76dd-4c8a-98de-1f850b110d44 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:29.401592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee584f5-76dd-4c8a-98de-1f850b110d44 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:29.403787 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:57:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:29.411974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973451b-abe4-48cf-9b7f-f908cf93b4b2 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:29.412442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973451b-abe4-48cf-9b7f-f908cf93b4b2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:29.412534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973451b-abe4-48cf-9b7f-f908cf93b4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:29.490796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-d973451b-abe4-48cf-9b7f-f908cf93b4b2 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:29.492047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d973451b-abe4-48cf-9b7f-f908cf93b4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:29.502557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:29.604571 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:29.643635 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee584f5-76dd-4c8a-98de-1f850b110d44 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:29.645215 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9ee584f5-76dd-4c8a-98de-1f850b110d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:29.776229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee584f5-76dd-4c8a-98de-1f850b110d44 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:29.839010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:29.861488 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28ba51-64d8-4816-b33c-66eacc6664c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:29.862209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28ba51-64d8-4816-b33c-66eacc6664c9 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:29.862565 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28ba51-64d8-4816-b33c-66eacc6664c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:29.862888 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28ba51-64d8-4816-b33c-66eacc6664c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:29.864650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ef28ba51-64d8-4816-b33c-66eacc6664c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:29.989243 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28ba51-64d8-4816-b33c-66eacc6664c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:29.991793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28ba51-64d8-4816-b33c-66eacc6664c9 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:30.015563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 740/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:32.205024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6879ca5-e46e-49bd-aec9-e710f20ea310 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:32.205605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6879ca5-e46e-49bd-aec9-e710f20ea310 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:32.205734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6879ca5-e46e-49bd-aec9-e710f20ea310 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:32.207188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:32.212801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68607699-1542-43f9-83e9-8d5bd38bc879 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:32.213252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68607699-1542-43f9-83e9-8d5bd38bc879 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:32.213347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68607699-1542-43f9-83e9-8d5bd38bc879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:32.296689 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-68607699-1542-43f9-83e9-8d5bd38bc879 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:32.297935 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-68607699-1542-43f9-83e9-8d5bd38bc879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:32.338944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:32.400507 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:32.439079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6879ca5-e46e-49bd-aec9-e710f20ea310 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:32.440640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b6879ca5-e46e-49bd-aec9-e710f20ea310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:32.550428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6879ca5-e46e-49bd-aec9-e710f20ea310 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:32.589278 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:32.630242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5c9d3-cb7b-4040-8293-fb10cca7c8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:32.630927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5c9d3-cb7b-4040-8293-fb10cca7c8af service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:32.631256 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5c9d3-cb7b-4040-8293-fb10cca7c8af service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:32.631585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5c9d3-cb7b-4040-8293-fb10cca7c8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:32.633263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e8b5c9d3-cb7b-4040-8293-fb10cca7c8af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:32.748208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5c9d3-cb7b-4040-8293-fb10cca7c8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:32.749583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5c9d3-cb7b-4040-8293-fb10cca7c8af service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:32.770147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 723/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:34.877018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c0f80-4773-41d7-a09b-7f22f89d2c60 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:34.877599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c0f80-4773-41d7-a09b-7f22f89d2c60 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:34.877599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c0f80-4773-41d7-a09b-7f22f89d2c60 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:34.879078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:34.884532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d573bd4-136c-46f4-95d4-442e799c60b9 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:34.884969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d573bd4-136c-46f4-95d4-442e799c60b9 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:34.885092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d573bd4-136c-46f4-95d4-442e799c60b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:34.957189 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3d573bd4-136c-46f4-95d4-442e799c60b9 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:34.958471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3d573bd4-136c-46f4-95d4-442e799c60b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:34.977250 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:35.062140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:57:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:35.100593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c0f80-4773-41d7-a09b-7f22f89d2c60 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:35.102159 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-970c0f80-4773-41d7-a09b-7f22f89d2c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:35.220711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c0f80-4773-41d7-a09b-7f22f89d2c60 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:35.291930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:35.299765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52010dcf-e3b4-4e58-9492-e6a5250f4e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:35.300430 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52010dcf-e3b4-4e58-9492-e6a5250f4e12 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:35.300741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52010dcf-e3b4-4e58-9492-e6a5250f4e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:35.301021 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52010dcf-e3b4-4e58-9492-e6a5250f4e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:35.302717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-52010dcf-e3b4-4e58-9492-e6a5250f4e12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:35.420388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52010dcf-e3b4-4e58-9492-e6a5250f4e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:35.421766 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52010dcf-e3b4-4e58-9492-e6a5250f4e12 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:35.442660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 743/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:37.519002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbd370-f3d3-4942-a9c7-a3bf9a1c07e8 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:37.519647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbd370-f3d3-4942-a9c7-a3bf9a1c07e8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:37.519647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbd370-f3d3-4942-a9c7-a3bf9a1c07e8 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:37.520961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 18:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:37.526685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5132f89-dd42-4e0a-860a-fc6893b8785e None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:37.527212 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5132f89-dd42-4e0a-860a-fc6893b8785e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:37.527321 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5132f89-dd42-4e0a-860a-fc6893b8785e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:37.596345 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-b5132f89-dd42-4e0a-860a-fc6893b8785e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:37.597590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b5132f89-dd42-4e0a-860a-fc6893b8785e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:37.651204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:37.701534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 18:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:57:37.739077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbd370-f3d3-4942-a9c7-a3bf9a1c07e8 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:57:37.740632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-febbd370-f3d3-4942-a9c7-a3bf9a1c07e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:37.853530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbd370-f3d3-4942-a9c7-a3bf9a1c07e8 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:57:37.902852 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:37.929499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909f1f2-7607-4d63-b8b0-315ae99074b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:57:37.930155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909f1f2-7607-4d63-b8b0-315ae99074b0 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:37.930476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909f1f2-7607-4d63-b8b0-315ae99074b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:37.930757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909f1f2-7607-4d63-b8b0-315ae99074b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:37.932410 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1909f1f2-7607-4d63-b8b0-315ae99074b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:38.053648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909f1f2-7607-4d63-b8b0-315ae99074b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:57:38.055127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909f1f2-7607-4d63-b8b0-315ae99074b0 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:57:38.076559 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 726/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:57:47.699679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=22 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:48.074359 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=66 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:57:59.688144 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecf76b-2d78-46ac-976a-a91a1b2c6926 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:59.688979 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecf76b-2d78-46ac-976a-a91a1b2c6926 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:59.688979 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecf76b-2d78-46ac-976a-a91a1b2c6926 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:59.690126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 745/1471] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:57:59.698053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a40690-bba2-4fe2-a32a-c8031d734ac1 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:57:59.698904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a40690-bba2-4fe2-a32a-c8031d734ac1 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:57:59.698904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a40690-bba2-4fe2-a32a-c8031d734ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:57:59.786518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-55a40690-bba2-4fe2-a32a-c8031d734ac1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:57:59.787928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-55a40690-bba2-4fe2-a32a-c8031d734ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:57:59.935216 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 727/1472] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:58:00.036149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1413f6-3d36-41c6-a5b8-919ab068116a None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:00.036445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1413f6-3d36-41c6-a5b8-919ab068116a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:00.036739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1413f6-3d36-41c6-a5b8-919ab068116a None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:00.038160 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 746/1473] 10.222.0.17 () {54 vars in 966 bytes} [Mon Jul 25 18:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:58:00.044486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f21d97-564d-4056-967d-13e0b589a215 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:00.044776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f21d97-564d-4056-967d-13e0b589a215 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:00.045093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f21d97-564d-4056-967d-13e0b589a215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:00.162422 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b4f21d97-564d-4056-967d-13e0b589a215 None None] MFA Rules not processed for user `28954d67689a46ee975fdfc9139e41eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:58:00.164143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b4f21d97-564d-4056-967d-13e0b589a215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:00.194848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:00.267163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 728/1474] 10.222.0.17 () {58 vars in 1063 bytes} [Mon Jul 25 18:58:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 18:58:00.279102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1413f6-3d36-41c6-a5b8-919ab068116a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:58:00.280685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3c1413f6-3d36-41c6-a5b8-919ab068116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:00.425109 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1413f6-3d36-41c6-a5b8-919ab068116a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:58:00.525979 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603896b7-0d9c-46a4-8a68-22bf7d21581e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28954d67689a46ee975fdfc9139e41eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:58:00.533214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603896b7-0d9c-46a4-8a68-22bf7d21581e service glance] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:00.533214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603896b7-0d9c-46a4-8a68-22bf7d21581e service glance] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:00.533214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603896b7-0d9c-46a4-8a68-22bf7d21581e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:00.533214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-603896b7-0d9c-46a4-8a68-22bf7d21581e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:00.619668 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:00.700729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603896b7-0d9c-46a4-8a68-22bf7d21581e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:58:00.702196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603896b7-0d9c-46a4-8a68-22bf7d21581e service glance] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:58:00.724053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 747/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:58:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:58:03.156128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d408ea-3f8d-4c88-81e0-da8f9b39e9d2 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:03.156805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d408ea-3f8d-4c88-81e0-da8f9b39e9d2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:03.156805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d408ea-3f8d-4c88-81e0-da8f9b39e9d2 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:03.158158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 729/1476] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:58:03.164841 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb5049-2163-4314-ae38-414a9f1740c5 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:03.165289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb5049-2163-4314-ae38-414a9f1740c5 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:03.165396 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb5049-2163-4314-ae38-414a9f1740c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:03.252243 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-aadb5049-2163-4314-ae38-414a9f1740c5 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:58:03.253375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-aadb5049-2163-4314-ae38-414a9f1740c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:03.370729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 748/1477] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:58:03.398061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d408ea-3f8d-4c88-81e0-da8f9b39e9d2 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:58:03.399671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-38d408ea-3f8d-4c88-81e0-da8f9b39e9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:03.557492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d408ea-3f8d-4c88-81e0-da8f9b39e9d2 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:58:03.618746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:03.703096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae38e1-ec70-4a60-8c84-4845f961bbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:58:03.703379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae38e1-ec70-4a60-8c84-4845f961bbff admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:03.703606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae38e1-ec70-4a60-8c84-4845f961bbff admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:03.703881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae38e1-ec70-4a60-8c84-4845f961bbff admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:03.704637 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae38e1-ec70-4a60-8c84-4845f961bbff admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:58:03.706880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: WARNING py.warnings [None req-b2ae38e1-ec70-4a60-8c84-4845f961bbff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 18:58:03.706880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: warnings.warn(msg) Jul 25 18:58:03.706880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: Jul 25 18:58:03.707646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae38e1-ec70-4a60-8c84-4845f961bbff admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:58:03.732201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 730/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 18:58:03 2022] GET /identity/v3/projects => generated 2553 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 18:58:05.921893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf0114-c526-4d00-b106-8b0a13b61b0b None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:05.922479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf0114-c526-4d00-b106-8b0a13b61b0b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:05.922479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf0114-c526-4d00-b106-8b0a13b61b0b None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:05.923954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 749/1479] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:58:05.929991 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5191c-90a7-4096-bfa3-a7a3465071a3 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:05.930602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5191c-90a7-4096-bfa3-a7a3465071a3 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:05.930602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5191c-90a7-4096-bfa3-a7a3465071a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:06.015376 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-93c5191c-90a7-4096-bfa3-a7a3465071a3 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:58:06.016626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-93c5191c-90a7-4096-bfa3-a7a3465071a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:06.132788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 731/1480] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:58:06.188036 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf0114-c526-4d00-b106-8b0a13b61b0b None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:58:06.189622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ccbf0114-c526-4d00-b106-8b0a13b61b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:06.201209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:06.371214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf0114-c526-4d00-b106-8b0a13b61b0b None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:58:06.466039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:06.514128 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9c951-0215-43d3-8c37-3978f83af601 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:58:06.514128 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9c951-0215-43d3-8c37-3978f83af601 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:06.514616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9c951-0215-43d3-8c37-3978f83af601 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:06.514775 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9c951-0215-43d3-8c37-3978f83af601 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:06.516453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5ea9c951-0215-43d3-8c37-3978f83af601 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:06.629111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9c951-0215-43d3-8c37-3978f83af601 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:58:06.630761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9c951-0215-43d3-8c37-3978f83af601 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:58:06.652725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 750/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:58:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:58:09.007646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fc428-4009-4ce4-a13e-fd613086a5f7 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:09.008258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fc428-4009-4ce4-a13e-fd613086a5f7 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:09.008258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fc428-4009-4ce4-a13e-fd613086a5f7 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:09.009620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 732/1482] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:58:09.015335 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a208d-4fa1-4dbb-81c9-d40182450750 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:09.015806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a208d-4fa1-4dbb-81c9-d40182450750 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:09.015914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a208d-4fa1-4dbb-81c9-d40182450750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:09.085125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-621a208d-4fa1-4dbb-81c9-d40182450750 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:58:09.086371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-621a208d-4fa1-4dbb-81c9-d40182450750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:09.136344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:09.191233 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 751/1483] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:58:09.227855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fc428-4009-4ce4-a13e-fd613086a5f7 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:58:09.230411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3f9fc428-4009-4ce4-a13e-fd613086a5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:09.256897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:09.342610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fc428-4009-4ce4-a13e-fd613086a5f7 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:58:09.424489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07036c-57f7-4d1e-a467-8b57838e0237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:58:09.425151 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07036c-57f7-4d1e-a467-8b57838e0237 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:09.425443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07036c-57f7-4d1e-a467-8b57838e0237 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:09.425723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07036c-57f7-4d1e-a467-8b57838e0237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:09.427552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6a07036c-57f7-4d1e-a467-8b57838e0237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:09.507377 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:09.544453 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a07036c-57f7-4d1e-a467-8b57838e0237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:58:09.545902 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a07036c-57f7-4d1e-a467-8b57838e0237 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:58:09.565896 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 733/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:58:11.723384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ca49f-a6fd-4f1d-b676-bd00865758ca None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:11.723975 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ca49f-a6fd-4f1d-b676-bd00865758ca None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:11.724175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ca49f-a6fd-4f1d-b676-bd00865758ca None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:11.725676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 752/1485] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:58:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:58:11.731622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c59e58-d91c-4d8a-b4c3-cf528fd261fe None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:11.732139 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c59e58-d91c-4d8a-b4c3-cf528fd261fe None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:11.732275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c59e58-d91c-4d8a-b4c3-cf528fd261fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:11.801757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-44c59e58-d91c-4d8a-b4c3-cf528fd261fe None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:58:11.802988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-44c59e58-d91c-4d8a-b4c3-cf528fd261fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:11.904439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 734/1486] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:58:11.943023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ca49f-a6fd-4f1d-b676-bd00865758ca None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:58:11.944595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f69ca49f-a6fd-4f1d-b676-bd00865758ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:12.055355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ca49f-a6fd-4f1d-b676-bd00865758ca None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:58:12.129692 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:12.131534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff241a82-f92b-4e8a-9c45-cc5e1fc22b93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:58:12.132252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff241a82-f92b-4e8a-9c45-cc5e1fc22b93 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:12.132540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff241a82-f92b-4e8a-9c45-cc5e1fc22b93 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:12.132821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff241a82-f92b-4e8a-9c45-cc5e1fc22b93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:12.134533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ff241a82-f92b-4e8a-9c45-cc5e1fc22b93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:12.258777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff241a82-f92b-4e8a-9c45-cc5e1fc22b93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:58:12.260632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff241a82-f92b-4e8a-9c45-cc5e1fc22b93 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:58:12.280232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 753/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:58:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 18:58:14.507679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f515c42-b426-4b44-9fcd-a1479eb535f1 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:14.508261 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f515c42-b426-4b44-9fcd-a1479eb535f1 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:14.508261 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f515c42-b426-4b44-9fcd-a1479eb535f1 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:14.509734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 735/1488] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:58:14.515479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e692be9-f036-4ffe-a483-1f57da768fef None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:14.515942 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e692be9-f036-4ffe-a483-1f57da768fef None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:14.516070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e692be9-f036-4ffe-a483-1f57da768fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:14.590561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-9e692be9-f036-4ffe-a483-1f57da768fef None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:58:14.591811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9e692be9-f036-4ffe-a483-1f57da768fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:14.650552 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:14.696309 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 754/1489] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:58:14.737789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f515c42-b426-4b44-9fcd-a1479eb535f1 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:58:14.739391 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1f515c42-b426-4b44-9fcd-a1479eb535f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:14.778316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:14.850979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f515c42-b426-4b44-9fcd-a1479eb535f1 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:58:14.927924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0445d-a0ea-441f-b852-666ba5a06652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:58:14.928588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0445d-a0ea-441f-b852-666ba5a06652 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:14.928875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0445d-a0ea-441f-b852-666ba5a06652 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:14.929155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0445d-a0ea-441f-b852-666ba5a06652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:14.930864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e7e0445d-a0ea-441f-b852-666ba5a06652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:15.019246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:15.055221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0445d-a0ea-441f-b852-666ba5a06652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:58:15.087487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0445d-a0ea-441f-b852-666ba5a06652 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:58:15.087487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 736/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:58:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 18:58:17.772047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368333a3-ea1b-44b7-963c-9b67433f99fb None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:17.772634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368333a3-ea1b-44b7-963c-9b67433f99fb None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:17.772634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368333a3-ea1b-44b7-963c-9b67433f99fb None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:17.774015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 755/1491] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 18:58:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 18:58:17.780127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014ab7c-4353-4301-ac88-9648c1e8616f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:17.780594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014ab7c-4353-4301-ac88-9648c1e8616f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:17.780704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014ab7c-4353-4301-ac88-9648c1e8616f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:17.851672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-8014ab7c-4353-4301-ac88-9648c1e8616f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 18:58:17.852911 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8014ab7c-4353-4301-ac88-9648c1e8616f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:17.954412 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 737/1492] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 18:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 18:58:17.995611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368333a3-ea1b-44b7-963c-9b67433f99fb None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 18:58:17.997169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-368333a3-ea1b-44b7-963c-9b67433f99fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:18.109207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368333a3-ea1b-44b7-963c-9b67433f99fb None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 18:58:18.190349 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cc8dd-2710-4da0-8910-3d1b98c07744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 18:58:18.191008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cc8dd-2710-4da0-8910-3d1b98c07744 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 18:58:18.191297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cc8dd-2710-4da0-8910-3d1b98c07744 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 18:58:18.191593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cc8dd-2710-4da0-8910-3d1b98c07744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 18:58:18.193372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5e5cc8dd-2710-4da0-8910-3d1b98c07744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 18:58:18.203421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:18.311986 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cc8dd-2710-4da0-8910-3d1b98c07744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 18:58:18.313435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cc8dd-2710-4da0-8910-3d1b98c07744 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 18:58:18.332305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 756/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 18:58:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 18:58:27.952373 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=82 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 18:58:28.329583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=50 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:00:01.443081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c08f6-168d-4ce5-962c-556438aaa0fd None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:01.443687 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c08f6-168d-4ce5-962c-556438aaa0fd None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:01.443687 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c08f6-168d-4ce5-962c-556438aaa0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:01.542414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-315c08f6-168d-4ce5-962c-556438aaa0fd None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:00:01.543666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-315c08f6-168d-4ce5-962c-556438aaa0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:00:01.657131 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 738/1494] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:00:01.665840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7757f-37f0-41dd-b352-5823226a8b26 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:01.666433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7757f-37f0-41dd-b352-5823226a8b26 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:01.666536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7757f-37f0-41dd-b352-5823226a8b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:01.686543 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-89c7757f-37f0-41dd-b352-5823226a8b26 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:00:01.687764 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-89c7757f-37f0-41dd-b352-5823226a8b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:00:01.696610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 757/1495] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:00:01.701318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c08f6-168d-4ce5-962c-556438aaa0fd None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:00:01.711654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c08f6-168d-4ce5-962c-556438aaa0fd None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:00:01.717132 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cfc00-0f56-4d2e-834e-1f109fa4a22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:00:01.717802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cfc00-0f56-4d2e-834e-1f109fa4a22b admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:01.718089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cfc00-0f56-4d2e-834e-1f109fa4a22b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:01.718408 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cfc00-0f56-4d2e-834e-1f109fa4a22b admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:01.719243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cfc00-0f56-4d2e-834e-1f109fa4a22b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:00:01.719565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cfc00-0f56-4d2e-834e-1f109fa4a22b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:00:01.720592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: WARNING py.warnings [None req-2e2cfc00-0f56-4d2e-834e-1f109fa4a22b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:00:01.720592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: warnings.warn(msg) Jul 25 19:00:01.720592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: Jul 25 19:00:01.721474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cfc00-0f56-4d2e-834e-1f109fa4a22b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:00:01.734434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 739/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:00:01.744497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7757f-37f0-41dd-b352-5823226a8b26 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:00:01.754682 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7757f-37f0-41dd-b352-5823226a8b26 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:00:01.760143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602f93d-a601-4cd0-92e1-04b1cd1ebedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:00:01.760929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f93d-a601-4cd0-92e1-04b1cd1ebedd admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:01.761110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f93d-a601-4cd0-92e1-04b1cd1ebedd admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:01.761372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f93d-a601-4cd0-92e1-04b1cd1ebedd admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:01.762066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602f93d-a601-4cd0-92e1-04b1cd1ebedd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:00:01.763901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: WARNING py.warnings [None req-1602f93d-a601-4cd0-92e1-04b1cd1ebedd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:00:01.763901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: warnings.warn(msg) Jul 25 19:00:01.763901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: Jul 25 19:00:01.765110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602f93d-a601-4cd0-92e1-04b1cd1ebedd admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:00:01.919841 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 758/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:00:01 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:00:01.924365 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cfc00-0f56-4d2e-834e-1f109fa4a22b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:00:01.935842 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cfc00-0f56-4d2e-834e-1f109fa4a22b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:00:01.994599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:00:02.001686 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fff39-0f68-407c-846b-f64e57435b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:00:02.002464 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fff39-0f68-407c-846b-f64e57435b6b admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:02.002867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fff39-0f68-407c-846b-f64e57435b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:02.003061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fff39-0f68-407c-846b-f64e57435b6b admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:02.003775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9fff39-0f68-407c-846b-f64e57435b6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:00:02.004897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: WARNING py.warnings [None req-eb9fff39-0f68-407c-846b-f64e57435b6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:00:02.004897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: warnings.warn(msg) Jul 25 19:00:02.004897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: Jul 25 19:00:02.006037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9fff39-0f68-407c-846b-f64e57435b6b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:00:02.104724 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 740/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:00:01 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:00:02.109352 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602f93d-a601-4cd0-92e1-04b1cd1ebedd admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:00:02.126912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602f93d-a601-4cd0-92e1-04b1cd1ebedd admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:00:02.132150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ffab3-8c22-4ed8-8055-b29cbfde8368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:00:02.132846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ffab3-8c22-4ed8-8055-b29cbfde8368 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:02.133133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ffab3-8c22-4ed8-8055-b29cbfde8368 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:02.133412 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ffab3-8c22-4ed8-8055-b29cbfde8368 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:02.134160 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ffab3-8c22-4ed8-8055-b29cbfde8368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:00:02.135272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: WARNING py.warnings [None req-866ffab3-8c22-4ed8-8055-b29cbfde8368 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:00:02.135272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: warnings.warn(msg) Jul 25 19:00:02.135272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: Jul 25 19:00:02.136174 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ffab3-8c22-4ed8-8055-b29cbfde8368 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:00:02.150494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 759/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:00:02.154761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fff39-0f68-407c-846b-f64e57435b6b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:00:02.165653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fff39-0f68-407c-846b-f64e57435b6b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:00:02.172276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0823035-932c-4960-8611-03b98e75edc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:00:02.172971 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0823035-932c-4960-8611-03b98e75edc7 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:02.173254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0823035-932c-4960-8611-03b98e75edc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:02.173531 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0823035-932c-4960-8611-03b98e75edc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:02.174257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0823035-932c-4960-8611-03b98e75edc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:00:02.175492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: WARNING py.warnings [None req-b0823035-932c-4960-8611-03b98e75edc7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:00:02.175492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: warnings.warn(msg) Jul 25 19:00:02.175492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: Jul 25 19:00:02.176232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0823035-932c-4960-8611-03b98e75edc7 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:00:02.192338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 741/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:00:02.196490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ffab3-8c22-4ed8-8055-b29cbfde8368 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:00:02.207651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ffab3-8c22-4ed8-8055-b29cbfde8368 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:00:02.212562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd277b-8de3-4d5d-ab64-c71e0e8b4735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:00:02.213510 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd277b-8de3-4d5d-ab64-c71e0e8b4735 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:02.213974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd277b-8de3-4d5d-ab64-c71e0e8b4735 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:02.214282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd277b-8de3-4d5d-ab64-c71e0e8b4735 admin admin] PATH_INFO: `/v3/projects/244a36a709ef4dc393b09071bd8a1df0/users/9f7d1fd660254669bed855d8660393e5/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:02.233748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd277b-8de3-4d5d-ab64-c71e0e8b4735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=244a36a709ef4dc393b09071bd8a1df0, user_id=9f7d1fd660254669bed855d8660393e5, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:00:02.235600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: WARNING py.warnings [None req-52dd277b-8de3-4d5d-ab64-c71e0e8b4735 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:00:02.235600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: warnings.warn(msg) Jul 25 19:00:02.235600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: Jul 25 19:00:02.237496 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd277b-8de3-4d5d-ab64-c71e0e8b4735 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:00:02.313743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 760/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:00:02 2022] PUT /identity/v3/projects/244a36a709ef4dc393b09071bd8a1df0/users/9f7d1fd660254669bed855d8660393e5/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:00:02.327039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885658c-b1ae-4daa-94c3-60f76d6f3f7c None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:02.327318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885658c-b1ae-4daa-94c3-60f76d6f3f7c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:02.327598 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885658c-b1ae-4daa-94c3-60f76d6f3f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:02.363694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-2885658c-b1ae-4daa-94c3-60f76d6f3f7c None None] MFA Rules not processed for user `9f7d1fd660254669bed855d8660393e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:00:02.363694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2885658c-b1ae-4daa-94c3-60f76d6f3f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:00:02.437701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 742/1502] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:00:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:00:02.452071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53968e3d-cfb0-4810-90d2-b5986df4fc25 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:02.452364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53968e3d-cfb0-4810-90d2-b5986df4fc25 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:02.452649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53968e3d-cfb0-4810-90d2-b5986df4fc25 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:02.454115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 761/1503] 10.222.0.17 () {54 vars in 964 bytes} [Mon Jul 25 19:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:00:02.459667 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a37bb3-2305-4567-8b11-bb80c5e70c32 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:02.460221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a37bb3-2305-4567-8b11-bb80c5e70c32 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:02.460221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a37bb3-2305-4567-8b11-bb80c5e70c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:02.531415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-08a37bb3-2305-4567-8b11-bb80c5e70c32 None None] MFA Rules not processed for user `9a419d02a7dc46e9b3c5b3d4eb3151d6`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:00:02.532634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-08a37bb3-2305-4567-8b11-bb80c5e70c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:00:02.622138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 743/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Mon Jul 25 19:00:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:00:02.630144 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53968e3d-cfb0-4810-90d2-b5986df4fc25 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:00:02.642730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53968e3d-cfb0-4810-90d2-b5986df4fc25 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:00:02.646916 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec719f7-7d65-4289-a458-7802612eee40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:00:02.647579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec719f7-7d65-4289-a458-7802612eee40 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:02.647870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec719f7-7d65-4289-a458-7802612eee40 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:02.648153 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec719f7-7d65-4289-a458-7802612eee40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:02.658228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec719f7-7d65-4289-a458-7802612eee40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:00:02.659728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec719f7-7d65-4289-a458-7802612eee40 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:00:02.669490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 762/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:00:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:00:02.716541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3381580-b3c7-4a2a-b7ca-12ec831b5b77 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:02.716838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3381580-b3c7-4a2a-b7ca-12ec831b5b77 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:02.717120 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3381580-b3c7-4a2a-b7ca-12ec831b5b77 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:02.718466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 744/1506] 10.222.0.17 () {52 vars in 830 bytes} [Mon Jul 25 19:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:00:02.722065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec719f7-7d65-4289-a458-7802612eee40 service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:00:02.788791 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec719f7-7d65-4289-a458-7802612eee40 service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:00:02.794047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:00:02.794966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:02.795252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:02.795537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] PATH_INFO: `/v3/users/9f7d1fd660254669bed855d8660393e5` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:02.796765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f7d1fd660254669bed855d8660393e5)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:00:02.797809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: WARNING py.warnings [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:00:02.797809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: warnings.warn(msg) Jul 25 19:00:02.797809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: Jul 25 19:00:02.799085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:00:03.034571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f7d1fd660254669bed855d8660393e5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:00:03.035298 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f7d1fd660254669bed855d8660393e5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:00:03.048526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f7d1fd660254669bed855d8660393e5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:00:03.067899 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f7d1fd660254669bed855d8660393e5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:00:03.071890 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:00:03.177579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f7d1fd660254669bed855d8660393e5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:00:03.184233 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 763/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:00:02 2022] DELETE /identity/v3/users/9f7d1fd660254669bed855d8660393e5 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:00:03.189127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680abf4-fbb0-45ac-9826-b6a32c099232 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:03.189424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680abf4-fbb0-45ac-9826-b6a32c099232 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:03.189738 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680abf4-fbb0-45ac-9826-b6a32c099232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:03.209535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e680abf4-fbb0-45ac-9826-b6a32c099232 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:00:03.210791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e680abf4-fbb0-45ac-9826-b6a32c099232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:00:03.220948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:00:03.272166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 745/1508] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:00:03.282511 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:00:03.288602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:00:03.367951 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4ec5f-af05-4a3f-8d08-9a83d7ae2020 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:00:03.371882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64b9cc-ce5c-4e0c-a0eb-185f0f06458b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:00:03.372555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64b9cc-ce5c-4e0c-a0eb-185f0f06458b service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:03.372842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64b9cc-ce5c-4e0c-a0eb-185f0f06458b service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:03.373131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64b9cc-ce5c-4e0c-a0eb-185f0f06458b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:03.383367 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64b9cc-ce5c-4e0c-a0eb-185f0f06458b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:00:03.384835 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64b9cc-ce5c-4e0c-a0eb-185f0f06458b service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:00:03.395588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 764/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:00:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:00:03.905817 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680abf4-fbb0-45ac-9826-b6a32c099232 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:00:03.907393 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e680abf4-fbb0-45ac-9826-b6a32c099232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:00:03.920632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680abf4-fbb0-45ac-9826-b6a32c099232 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:00:03.925803 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:00:03.926731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:00:03.927019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:00:03.927298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c admin admin] PATH_INFO: `/v3/projects/244a36a709ef4dc393b09071bd8a1df0` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:00:03.928499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=244a36a709ef4dc393b09071bd8a1df0)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:00:03.929517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: WARNING py.warnings [None req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:00:03.929517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: warnings.warn(msg) Jul 25 19:00:03.929517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: Jul 25 19:00:03.930765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:00:04.039704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '244a36a709ef4dc393b09071bd8a1df0', 'request_id': 'req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:00:04.149109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '244a36a709ef4dc393b09071bd8a1df0', 'request_id': 'req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:00:04.156987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '244a36a709ef4dc393b09071bd8a1df0', 'request_id': 'req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:00:04.178544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c admin admin] The token cache is being invalidate because project 244a36a709ef4dc393b09071bd8a1df0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:00:04.178843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:00:04.179623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-081a577c-af7e-4d1b-8630-d4b51f9d6e8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:00:04.181201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 746/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:00:03 2022] DELETE /identity/v3/projects/244a36a709ef4dc393b09071bd8a1df0 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:00:13.390813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:00:14.175239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:01:54.724564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1decb-6565-4ae1-b72e-a0df743b2f9d None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:01:54.725194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1decb-6565-4ae1-b72e-a0df743b2f9d None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:01:54.725194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1decb-6565-4ae1-b72e-a0df743b2f9d None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:01:54.726641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:01:54.732701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ae5a6-f54e-4915-be3e-1a09d911f946 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:01:54.733472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ae5a6-f54e-4915-be3e-1a09d911f946 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:01:54.733472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ae5a6-f54e-4915-be3e-1a09d911f946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:01:54.754676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ad2ae5a6-f54e-4915-be3e-1a09d911f946 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:01:54.755923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ad2ae5a6-f54e-4915-be3e-1a09d911f946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:01:54.824105 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:01:54.981758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1decb-6565-4ae1-b72e-a0df743b2f9d None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:01:54.983859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-78d1decb-6565-4ae1-b72e-a0df743b2f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:01:55.102068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1decb-6565-4ae1-b72e-a0df743b2f9d None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:01:55.108051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd97bc1-22f4-42f1-a71c-942601eb7207 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:01:55.109009 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd97bc1-22f4-42f1-a71c-942601eb7207 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:01:55.109399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd97bc1-22f4-42f1-a71c-942601eb7207 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:01:55.109784 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd97bc1-22f4-42f1-a71c-942601eb7207 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:01:55.120839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd97bc1-22f4-42f1-a71c-942601eb7207 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:01:55.122520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd97bc1-22f4-42f1-a71c-942601eb7207 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:01:55.135180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 766/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:01:56.045424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b380d-e615-4dcd-9a1e-3602a1263976 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:01:56.046162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b380d-e615-4dcd-9a1e-3602a1263976 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:01:56.046471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b380d-e615-4dcd-9a1e-3602a1263976 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:01:56.049207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:01:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:01:56.055993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91f2b5-b5d0-4a16-bf5a-9f15fbd266cd None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:01:56.056626 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91f2b5-b5d0-4a16-bf5a-9f15fbd266cd None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:01:56.056626 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91f2b5-b5d0-4a16-bf5a-9f15fbd266cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:01:56.081107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ab91f2b5-b5d0-4a16-bf5a-9f15fbd266cd None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:01:56.082395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ab91f2b5-b5d0-4a16-bf5a-9f15fbd266cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:01:56.091861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:01:56.253085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b380d-e615-4dcd-9a1e-3602a1263976 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:01:56.274910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b380d-e615-4dcd-9a1e-3602a1263976 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:01:56.280923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd042eac-e4e8-4b2d-9f03-4673499ac8bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:01:56.281598 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd042eac-e4e8-4b2d-9f03-4673499ac8bb service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:01:56.281921 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd042eac-e4e8-4b2d-9f03-4673499ac8bb service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:01:56.282205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd042eac-e4e8-4b2d-9f03-4673499ac8bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:01:56.292145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd042eac-e4e8-4b2d-9f03-4673499ac8bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:01:56.293900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd042eac-e4e8-4b2d-9f03-4673499ac8bb service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:01:56.306364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 749/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:01:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:01:57.995388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad54b63-e469-42dc-95c8-f52bd30838a6 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:01:57.996001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad54b63-e469-42dc-95c8-f52bd30838a6 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:01:57.996001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad54b63-e469-42dc-95c8-f52bd30838a6 None None] PATH_INFO: `/v3` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:01:57.997293 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 768/1517] 10.222.0.88 () {54 vars in 925 bytes} [Mon Jul 25 19:01:57 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:01:58.009781 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b725a00-11d5-4463-af3f-56a1e3e2a2af None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:01:58.010434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b725a00-11d5-4463-af3f-56a1e3e2a2af None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:01:58.010434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b725a00-11d5-4463-af3f-56a1e3e2a2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:01:58.070987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-8b725a00-11d5-4463-af3f-56a1e3e2a2af None None] MFA Rules not processed for user `f5027a60da214afda9144b5ae6cce15b`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:01:58.072165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8b725a00-11d5-4463-af3f-56a1e3e2a2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:01:58.160518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 750/1518] 10.222.0.88 () {58 vars in 1015 bytes} [Mon Jul 25 19:01:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:02:02.943779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c3382-2b70-43a0-b034-ecd5d7ef0fb7 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:02.944351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c3382-2b70-43a0-b034-ecd5d7ef0fb7 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:02.944351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c3382-2b70-43a0-b034-ecd5d7ef0fb7 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:02.946611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 769/1519] 10.222.0.17 () {54 vars in 961 bytes} [Mon Jul 25 19:02:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:02:02.952449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef1f58-5406-475d-a5e2-a69ecedd2b33 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:02.952949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef1f58-5406-475d-a5e2-a69ecedd2b33 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:02.953183 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef1f58-5406-475d-a5e2-a69ecedd2b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:02.978089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-fbef1f58-5406-475d-a5e2-a69ecedd2b33 None None] MFA Rules not processed for user `f5027a60da214afda9144b5ae6cce15b`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:02:02.979374 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-fbef1f58-5406-475d-a5e2-a69ecedd2b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:02:02.989243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 751/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Mon Jul 25 19:02:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:02:02.997470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c3382-2b70-43a0-b034-ecd5d7ef0fb7 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:02:03.016821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c3382-2b70-43a0-b034-ecd5d7ef0fb7 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:02:03.022454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10d44e-a954-435e-a9f6-7c325d9e503f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:02:03.023119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10d44e-a954-435e-a9f6-7c325d9e503f service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:03.023433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10d44e-a954-435e-a9f6-7c325d9e503f service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:03.024393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10d44e-a954-435e-a9f6-7c325d9e503f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:03.033912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10d44e-a954-435e-a9f6-7c325d9e503f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:02:03.035553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10d44e-a954-435e-a9f6-7c325d9e503f service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:02:03.046638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 770/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:02:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:02:07.605448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fe0d3-bdf3-4c83-b2dd-8a6a5ec13446 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:07.606141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fe0d3-bdf3-4c83-b2dd-8a6a5ec13446 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:07.606141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fe0d3-bdf3-4c83-b2dd-8a6a5ec13446 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:07.607732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:02:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:02:07.613828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2085c76-3c4b-4814-9629-62009d06f4ca None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:07.614359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2085c76-3c4b-4814-9629-62009d06f4ca None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:07.614525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2085c76-3c4b-4814-9629-62009d06f4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:07.637729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d2085c76-3c4b-4814-9629-62009d06f4ca None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:02:07.639514 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d2085c76-3c4b-4814-9629-62009d06f4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:02:07.648037 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:02:07.811289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fe0d3-bdf3-4c83-b2dd-8a6a5ec13446 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:02:07.830697 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fe0d3-bdf3-4c83-b2dd-8a6a5ec13446 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:02:07.836992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162c987-62c6-4165-bf28-1ec241e74207 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:02:07.837687 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162c987-62c6-4165-bf28-1ec241e74207 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:07.837980 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162c987-62c6-4165-bf28-1ec241e74207 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:07.838352 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162c987-62c6-4165-bf28-1ec241e74207 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:07.849898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162c987-62c6-4165-bf28-1ec241e74207 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:02:07.851632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162c987-62c6-4165-bf28-1ec241e74207 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:02:07.863693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 753/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:02:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:02:08.761559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1c0f2-e16b-4760-9bd5-74412c796a01 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:08.762123 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1c0f2-e16b-4760-9bd5-74412c796a01 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:08.762244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1c0f2-e16b-4760-9bd5-74412c796a01 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:08.763927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:02:08.769574 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df960fe-fed3-47a0-a8c4-1c37372767f8 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:08.770054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df960fe-fed3-47a0-a8c4-1c37372767f8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:08.770054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df960fe-fed3-47a0-a8c4-1c37372767f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:08.791095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-5df960fe-fed3-47a0-a8c4-1c37372767f8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:02:08.792371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5df960fe-fed3-47a0-a8c4-1c37372767f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:02:08.800523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:02:08.955493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1c0f2-e16b-4760-9bd5-74412c796a01 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:02:08.971597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1c0f2-e16b-4760-9bd5-74412c796a01 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:02:08.978067 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d58bf-403e-4edd-8184-e327ae6cff14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:02:08.978926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d58bf-403e-4edd-8184-e327ae6cff14 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:08.979235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d58bf-403e-4edd-8184-e327ae6cff14 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:08.979618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d58bf-403e-4edd-8184-e327ae6cff14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:08.989268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d58bf-403e-4edd-8184-e327ae6cff14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:02:08.991021 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d58bf-403e-4edd-8184-e327ae6cff14 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:02:09.001488 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 773/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:02:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:02:13.426103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff011904-962a-49fa-8d9f-4fb8ce28f1f0 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:13.426687 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff011904-962a-49fa-8d9f-4fb8ce28f1f0 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:13.426687 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff011904-962a-49fa-8d9f-4fb8ce28f1f0 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:13.428033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:02:13.434216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65623aa7-c708-4e3d-af76-463642e172de None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:13.434715 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65623aa7-c708-4e3d-af76-463642e172de None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:13.434827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65623aa7-c708-4e3d-af76-463642e172de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:13.455277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-65623aa7-c708-4e3d-af76-463642e172de None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:02:13.456498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-65623aa7-c708-4e3d-af76-463642e172de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:02:13.464668 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:02:13.544235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff011904-962a-49fa-8d9f-4fb8ce28f1f0 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:02:13.545989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ff011904-962a-49fa-8d9f-4fb8ce28f1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:02:13.581012 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:02:13.620566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff011904-962a-49fa-8d9f-4fb8ce28f1f0 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:02:13.624601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f233c09-e645-43af-8143-f44135683cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:02:13.625364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f233c09-e645-43af-8143-f44135683cdb service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:13.625705 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f233c09-e645-43af-8143-f44135683cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:13.625988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f233c09-e645-43af-8143-f44135683cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:13.638245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f233c09-e645-43af-8143-f44135683cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:02:13.640170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f233c09-e645-43af-8143-f44135683cdb service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:02:13.650859 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 756/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:02:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:02:15.792781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9b941-87f3-4c4b-af67-34c72ccb90d9 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:15.792781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9b941-87f3-4c4b-af67-34c72ccb90d9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:15.792781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9b941-87f3-4c4b-af67-34c72ccb90d9 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:15.793661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:02:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:02:15.801548 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54050417-010c-40db-a32a-31db149e2516 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:15.802432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54050417-010c-40db-a32a-31db149e2516 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:15.802432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54050417-010c-40db-a32a-31db149e2516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:15.824099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-54050417-010c-40db-a32a-31db149e2516 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:02:15.825517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-54050417-010c-40db-a32a-31db149e2516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:02:15.836819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:02:15.875252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9b941-87f3-4c4b-af67-34c72ccb90d9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:02:15.886905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9b941-87f3-4c4b-af67-34c72ccb90d9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:02:15.892057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e78a7d5-aca7-477f-add8-0eec1ef78b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:02:15.893062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78a7d5-aca7-477f-add8-0eec1ef78b8d service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:15.893541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78a7d5-aca7-477f-add8-0eec1ef78b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:15.894007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78a7d5-aca7-477f-add8-0eec1ef78b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:15.907507 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e78a7d5-aca7-477f-add8-0eec1ef78b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:02:15.908966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e78a7d5-aca7-477f-add8-0eec1ef78b8d service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:02:15.921480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 776/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:02:20.250272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f2214-13c2-4ad9-b273-a6d1799569c3 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:20.251207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f2214-13c2-4ad9-b273-a6d1799569c3 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:20.251207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f2214-13c2-4ad9-b273-a6d1799569c3 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:20.252502 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:02:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:02:20.258842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440e8f9-6792-41ca-9e63-8a6936e98033 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:20.259707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440e8f9-6792-41ca-9e63-8a6936e98033 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:20.259707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440e8f9-6792-41ca-9e63-8a6936e98033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:20.282683 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-b440e8f9-6792-41ca-9e63-8a6936e98033 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:02:20.283952 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b440e8f9-6792-41ca-9e63-8a6936e98033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:02:20.293504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:02:20.625266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f2214-13c2-4ad9-b273-a6d1799569c3 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:02:20.627221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bb5f2214-13c2-4ad9-b273-a6d1799569c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:02:20.704222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f2214-13c2-4ad9-b273-a6d1799569c3 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:02:20.711670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e65035-bdfd-4203-b30d-87a314371af9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:02:20.717497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e65035-bdfd-4203-b30d-87a314371af9 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:20.717497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e65035-bdfd-4203-b30d-87a314371af9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:20.717497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e65035-bdfd-4203-b30d-87a314371af9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:20.724871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e65035-bdfd-4203-b30d-87a314371af9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:02:20.726429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e65035-bdfd-4203-b30d-87a314371af9 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:02:20.738558 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 759/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:02:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:02:22.911673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02cd31-b2c3-4cdd-ae13-ca20d1aa7fc9 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:22.912246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02cd31-b2c3-4cdd-ae13-ca20d1aa7fc9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:22.912246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02cd31-b2c3-4cdd-ae13-ca20d1aa7fc9 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:22.914365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:02:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:02:22.920239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b3f83-bd84-47a0-a8d6-849f66d29bab None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:22.920756 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b3f83-bd84-47a0-a8d6-849f66d29bab None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:22.920870 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b3f83-bd84-47a0-a8d6-849f66d29bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:22.945542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-292b3f83-bd84-47a0-a8d6-849f66d29bab None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:02:22.946905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-292b3f83-bd84-47a0-a8d6-849f66d29bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:02:22.962235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:02:23.280608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02cd31-b2c3-4cdd-ae13-ca20d1aa7fc9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:02:23.282615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3b02cd31-b2c3-4cdd-ae13-ca20d1aa7fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:02:23.297930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02cd31-b2c3-4cdd-ae13-ca20d1aa7fc9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:02:23.301870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659f96b-5190-432c-bc61-5abd9ea3279c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:02:23.302850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659f96b-5190-432c-bc61-5abd9ea3279c service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:02:23.303141 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659f96b-5190-432c-bc61-5abd9ea3279c service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:02:23.303813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659f96b-5190-432c-bc61-5abd9ea3279c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:02:23.314988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659f96b-5190-432c-bc61-5abd9ea3279c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:02:23.316497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659f96b-5190-432c-bc61-5abd9ea3279c service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:02:23.327405 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 779/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:02:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:02:32.939134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=47 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:02:33.322658 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=82 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:04.789731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2fa5d-950e-4ae5-aa16-29108c0bc7cf None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:04.790926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2fa5d-950e-4ae5-aa16-29108c0bc7cf None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:04.791529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2fa5d-950e-4ae5-aa16-29108c0bc7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:04.817302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-9aa2fa5d-950e-4ae5-aa16-29108c0bc7cf None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:04.822921 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9aa2fa5d-950e-4ae5-aa16-29108c0bc7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:04.832935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 761/1540] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:04.842451 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4410d-c678-4aa8-b51f-24f2a0eaa7bd None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:04.843603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4410d-c678-4aa8-b51f-24f2a0eaa7bd None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:04.844161 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4410d-c678-4aa8-b51f-24f2a0eaa7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:04.887468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-c4b4410d-c678-4aa8-b51f-24f2a0eaa7bd None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:04.889195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c4b4410d-c678-4aa8-b51f-24f2a0eaa7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:04.907292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 780/1541] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:04.912232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2fa5d-950e-4ae5-aa16-29108c0bc7cf None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:04.934495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2fa5d-950e-4ae5-aa16-29108c0bc7cf None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:04.946209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea156f-95dd-422c-9758-bf71c86e0962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:04.947322 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea156f-95dd-422c-9758-bf71c86e0962 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:04.947881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea156f-95dd-422c-9758-bf71c86e0962 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:04.948372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea156f-95dd-422c-9758-bf71c86e0962 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:04.949502 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea156f-95dd-422c-9758-bf71c86e0962 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:04:04.950002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea156f-95dd-422c-9758-bf71c86e0962 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:04.956547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea156f-95dd-422c-9758-bf71c86e0962 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:04.973293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 762/1542] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:04:04.982710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4410d-c678-4aa8-b51f-24f2a0eaa7bd None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:05.001421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4410d-c678-4aa8-b51f-24f2a0eaa7bd None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:05.009646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b400a-e50f-421d-ad3c-a0cba7eadfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:05.011122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b400a-e50f-421d-ad3c-a0cba7eadfda admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.011662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b400a-e50f-421d-ad3c-a0cba7eadfda admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.012167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b400a-e50f-421d-ad3c-a0cba7eadfda admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.013119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b400a-e50f-421d-ad3c-a0cba7eadfda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:05.015646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b400a-e50f-421d-ad3c-a0cba7eadfda admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:05.035717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f12c8-b116-4c33-a592-ecfe52871055 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.036401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f12c8-b116-4c33-a592-ecfe52871055 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.036905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f12c8-b116-4c33-a592-ecfe52871055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.072716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-8f3f12c8-b116-4c33-a592-ecfe52871055 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.074400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8f3f12c8-b116-4c33-a592-ecfe52871055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:05.084693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 763/1543] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:05.094600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac828a-fd55-4644-bae1-13380f3c7250 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.094937 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac828a-fd55-4644-bae1-13380f3c7250 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.095790 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac828a-fd55-4644-bae1-13380f3c7250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.119521 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-bdac828a-fd55-4644-bae1-13380f3c7250 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.121267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bdac828a-fd55-4644-bae1-13380f3c7250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:05.137762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 764/1544] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:04:05.146098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac828a-fd55-4644-bae1-13380f3c7250 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:05.150202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 781/1545] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:04 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:05.158375 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac828a-fd55-4644-bae1-13380f3c7250 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:05.164455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538bb69-4b38-4aa3-ba5e-95fa9065afe4 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.170597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538bb69-4b38-4aa3-ba5e-95fa9065afe4 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.170597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538bb69-4b38-4aa3-ba5e-95fa9065afe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.207487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-b538bb69-4b38-4aa3-ba5e-95fa9065afe4 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.209027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b538bb69-4b38-4aa3-ba5e-95fa9065afe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:05.264390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5a9f8-36ca-4c5a-8ae9-1fefb26b421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:05.265149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5a9f8-36ca-4c5a-8ae9-1fefb26b421c admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.265438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5a9f8-36ca-4c5a-8ae9-1fefb26b421c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.265776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5a9f8-36ca-4c5a-8ae9-1fefb26b421c admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.266563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5a9f8-36ca-4c5a-8ae9-1fefb26b421c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:04:05.266881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5a9f8-36ca-4c5a-8ae9-1fefb26b421c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:05.269275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5a9f8-36ca-4c5a-8ae9-1fefb26b421c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:05.290281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 765/1546] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:04:05.291217 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5a9f8-36ca-4c5a-8ae9-1fefb26b421c admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:05.312255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5a9f8-36ca-4c5a-8ae9-1fefb26b421c admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:05.326509 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455537b3-d543-4227-b50f-6403ec8be97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:05.328186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455537b3-d543-4227-b50f-6403ec8be97e admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.328811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455537b3-d543-4227-b50f-6403ec8be97e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.339033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455537b3-d543-4227-b50f-6403ec8be97e admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.340651 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455537b3-d543-4227-b50f-6403ec8be97e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:05.344086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455537b3-d543-4227-b50f-6403ec8be97e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:05.371562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 782/1547] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:04:05.373224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47c6eb-b6f0-402b-9c4c-dacf8ee204c8 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.373561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47c6eb-b6f0-402b-9c4c-dacf8ee204c8 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.373855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47c6eb-b6f0-402b-9c4c-dacf8ee204c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.398763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-5c47c6eb-b6f0-402b-9c4c-dacf8ee204c8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.399840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5c47c6eb-b6f0-402b-9c4c-dacf8ee204c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:05.409140 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 783/1548] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:05.409977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47c6eb-b6f0-402b-9c4c-dacf8ee204c8 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:05.421794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47c6eb-b6f0-402b-9c4c-dacf8ee204c8 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:05.428213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6f058-408f-484a-beb9-eb29c35bf96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:05.428934 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6f058-408f-484a-beb9-eb29c35bf96c admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.429217 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6f058-408f-484a-beb9-eb29c35bf96c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.429498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6f058-408f-484a-beb9-eb29c35bf96c admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.430193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6f058-408f-484a-beb9-eb29c35bf96c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:05.432147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6f058-408f-484a-beb9-eb29c35bf96c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:05.450269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 766/1549] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:05.451760 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb511950-1ea2-48df-88fe-55cee886a9ef None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.452061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb511950-1ea2-48df-88fe-55cee886a9ef None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.452342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb511950-1ea2-48df-88fe-55cee886a9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.476311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-bb511950-1ea2-48df-88fe-55cee886a9ef None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.477567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bb511950-1ea2-48df-88fe-55cee886a9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:05.486577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 767/1550] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:05.488007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799aebbb-511d-4820-9d87-7a4ae6dd432b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.488305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799aebbb-511d-4820-9d87-7a4ae6dd432b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.488666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799aebbb-511d-4820-9d87-7a4ae6dd432b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.510612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-799aebbb-511d-4820-9d87-7a4ae6dd432b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.512109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-799aebbb-511d-4820-9d87-7a4ae6dd432b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:05.522966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 768/1551] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:05.524258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90797383-a8f4-4a50-96e9-82417a48593b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.524550 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90797383-a8f4-4a50-96e9-82417a48593b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.524833 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90797383-a8f4-4a50-96e9-82417a48593b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.544569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 784/1552] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/projects => generated 391 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:05.545535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-90797383-a8f4-4a50-96e9-82417a48593b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.546115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751b9d4-2429-4ac5-be71-c95bf6755921 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.546646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751b9d4-2429-4ac5-be71-c95bf6755921 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.554546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-90797383-a8f4-4a50-96e9-82417a48593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:05.554714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751b9d4-2429-4ac5-be71-c95bf6755921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.570244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-6751b9d4-2429-4ac5-be71-c95bf6755921 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.571471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6751b9d4-2429-4ac5-be71-c95bf6755921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:05.626747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 769/1553] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:05.631544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdefdf6-7ae6-4476-8fcd-eebeeea8cdf7 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.631544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdefdf6-7ae6-4476-8fcd-eebeeea8cdf7 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.631544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdefdf6-7ae6-4476-8fcd-eebeeea8cdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.651489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 785/1554] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:05.652993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751b9d4-2429-4ac5-be71-c95bf6755921 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:05.653642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-9fdefdf6-7ae6-4476-8fcd-eebeeea8cdf7 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.655388 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9fdefdf6-7ae6-4476-8fcd-eebeeea8cdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:05.664860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751b9d4-2429-4ac5-be71-c95bf6755921 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:05.668961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 770/1555] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:05.670185 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e62656-f5d6-4896-9ad9-c21742d7fb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:05.670811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f1632-2bcd-4cf3-84c9-bd3477b945ec None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.670934 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e62656-f5d6-4896-9ad9-c21742d7fb8b admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.671147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f1632-2bcd-4cf3-84c9-bd3477b945ec None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.671247 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e62656-f5d6-4896-9ad9-c21742d7fb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.671438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f1632-2bcd-4cf3-84c9-bd3477b945ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.671608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e62656-f5d6-4896-9ad9-c21742d7fb8b admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.672313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e62656-f5d6-4896-9ad9-c21742d7fb8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:05.674293 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e62656-f5d6-4896-9ad9-c21742d7fb8b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:05.685874 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 786/1556] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:05.687275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9967e8-98b6-4108-ad0b-8cecfe892396 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.687568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9967e8-98b6-4108-ad0b-8cecfe892396 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.688492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9967e8-98b6-4108-ad0b-8cecfe892396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.695648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-9b3f1632-2bcd-4cf3-84c9-bd3477b945ec None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.696889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9b3f1632-2bcd-4cf3-84c9-bd3477b945ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:05.708413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 771/1557] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:05.708680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-be9967e8-98b6-4108-ad0b-8cecfe892396 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.709843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-be9967e8-98b6-4108-ad0b-8cecfe892396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:05.710173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6226c-4a81-44d1-a415-6cf25d313d11 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.710677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6226c-4a81-44d1-a415-6cf25d313d11 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.711165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6226c-4a81-44d1-a415-6cf25d313d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.723742 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 787/1558] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:05.726170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9967e8-98b6-4108-ad0b-8cecfe892396 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:05.734314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-98f6226c-4a81-44d1-a415-6cf25d313d11 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.735673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-98f6226c-4a81-44d1-a415-6cf25d313d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:05.737115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9967e8-98b6-4108-ad0b-8cecfe892396 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:05.745058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0790c8-7ea7-421a-a170-4075cb3bf702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:05.745836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0790c8-7ea7-421a-a170-4075cb3bf702 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.746128 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0790c8-7ea7-421a-a170-4075cb3bf702 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.746452 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0790c8-7ea7-421a-a170-4075cb3bf702 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.747252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0790c8-7ea7-421a-a170-4075cb3bf702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:05.747886 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 772/1559] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:05.748554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: WARNING py.warnings [None req-fd0790c8-7ea7-421a-a170-4075cb3bf702 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:04:05.748554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: warnings.warn(msg) Jul 25 19:04:05.748554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: Jul 25 19:04:05.749848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0790c8-7ea7-421a-a170-4075cb3bf702 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:05.752495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6226c-4a81-44d1-a415-6cf25d313d11 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:05.763612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6226c-4a81-44d1-a415-6cf25d313d11 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:05.769876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac28711-8a4e-44b2-8155-bf78f50dacb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:05.770652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac28711-8a4e-44b2-8155-bf78f50dacb0 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.770987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac28711-8a4e-44b2-8155-bf78f50dacb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.771270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac28711-8a4e-44b2-8155-bf78f50dacb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.772004 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac28711-8a4e-44b2-8155-bf78f50dacb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:04:05.772362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac28711-8a4e-44b2-8155-bf78f50dacb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:05.774270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac28711-8a4e-44b2-8155-bf78f50dacb0 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:05.782141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:05.788744 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 773/1560] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:04:05.790189 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157c2d1-304d-49f9-a73d-74ca82bc544b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.790892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157c2d1-304d-49f9-a73d-74ca82bc544b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.790892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157c2d1-304d-49f9-a73d-74ca82bc544b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.813254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-f157c2d1-304d-49f9-a73d-74ca82bc544b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.815401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f157c2d1-304d-49f9-a73d-74ca82bc544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:05.825964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 774/1561] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:05.827093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157c2d1-304d-49f9-a73d-74ca82bc544b None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:05.838314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157c2d1-304d-49f9-a73d-74ca82bc544b None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:05.843595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1502e-6cf5-481e-a481-6cc5ac967fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:05.844270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1502e-6cf5-481e-a481-6cc5ac967fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.844557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1502e-6cf5-481e-a481-6cc5ac967fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.844846 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1502e-6cf5-481e-a481-6cc5ac967fa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.845677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1502e-6cf5-481e-a481-6cc5ac967fa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:04:05.846072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1502e-6cf5-481e-a481-6cc5ac967fa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:05.847831 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1502e-6cf5-481e-a481-6cc5ac967fa5 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:05.937992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 775/1562] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:04:05.939971 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1502e-6cf5-481e-a481-6cc5ac967fa5 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:05.953541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1502e-6cf5-481e-a481-6cc5ac967fa5 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:05.959601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bced54d-31aa-47ca-9449-125eca247080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:05.960349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bced54d-31aa-47ca-9449-125eca247080 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.960725 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bced54d-31aa-47ca-9449-125eca247080 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.961075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bced54d-31aa-47ca-9449-125eca247080 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.961819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bced54d-31aa-47ca-9449-125eca247080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:05.963464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 788/1563] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/users => generated 296 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:05.966490 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bced54d-31aa-47ca-9449-125eca247080 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:05.966698 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9285b4a-8b70-43cb-9bba-cd7ba56e6972 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.966698 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9285b4a-8b70-43cb-9bba-cd7ba56e6972 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.968713 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9285b4a-8b70-43cb-9bba-cd7ba56e6972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.975966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 776/1564] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:05.976661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bced54d-31aa-47ca-9449-125eca247080 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:05.987236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bced54d-31aa-47ca-9449-125eca247080 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:05.993827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3dae9-2a30-415e-a0f1-f9fd87b940cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:05.994924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3dae9-2a30-415e-a0f1-f9fd87b940cc admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:05.995422 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3dae9-2a30-415e-a0f1-f9fd87b940cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:05.995915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3dae9-2a30-415e-a0f1-f9fd87b940cc admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:05.997413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3dae9-2a30-415e-a0f1-f9fd87b940cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:04:05.997832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-f9285b4a-8b70-43cb-9bba-cd7ba56e6972 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:05.998063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3dae9-2a30-415e-a0f1-f9fd87b940cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:05.999304 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f9285b4a-8b70-43cb-9bba-cd7ba56e6972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:06.000809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3dae9-2a30-415e-a0f1-f9fd87b940cc admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.009665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 789/1565] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:06.011118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9285b4a-8b70-43cb-9bba-cd7ba56e6972 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.017827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:06.018127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 777/1566] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:04:06.020746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3dae9-2a30-415e-a0f1-f9fd87b940cc admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.025629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9285b4a-8b70-43cb-9bba-cd7ba56e6972 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.030246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea0cff-0ee8-4ab1-94bb-2337a94ab686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.031000 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea0cff-0ee8-4ab1-94bb-2337a94ab686 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.031262 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea0cff-0ee8-4ab1-94bb-2337a94ab686 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.031545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea0cff-0ee8-4ab1-94bb-2337a94ab686 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.032280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea0cff-0ee8-4ab1-94bb-2337a94ab686 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:04:06.032626 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea0cff-0ee8-4ab1-94bb-2337a94ab686 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.032778 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3dae9-2a30-415e-a0f1-f9fd87b940cc admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.033672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: WARNING py.warnings [None req-29ea0cff-0ee8-4ab1-94bb-2337a94ab686 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:04:06.033672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: warnings.warn(msg) Jul 25 19:04:06.033672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: Jul 25 19:04:06.034706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea0cff-0ee8-4ab1-94bb-2337a94ab686 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.042657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa6af1-790f-40a5-bf0e-8cdcfaef35c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.044001 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa6af1-790f-40a5-bf0e-8cdcfaef35c6 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.044152 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa6af1-790f-40a5-bf0e-8cdcfaef35c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.044340 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa6af1-790f-40a5-bf0e-8cdcfaef35c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.045174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa6af1-790f-40a5-bf0e-8cdcfaef35c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.046166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: WARNING py.warnings [None req-c2aa6af1-790f-40a5-bf0e-8cdcfaef35c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:04:06.046166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: warnings.warn(msg) Jul 25 19:04:06.046166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: Jul 25 19:04:06.047519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa6af1-790f-40a5-bf0e-8cdcfaef35c6 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.050824 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 790/1567] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:04:06.053006 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea0cff-0ee8-4ab1-94bb-2337a94ab686 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.061909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea0cff-0ee8-4ab1-94bb-2337a94ab686 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.070536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6d34f-fe0b-4192-8bb7-98e29eca1f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.070827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6d34f-fe0b-4192-8bb7-98e29eca1f0e admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.071074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6d34f-fe0b-4192-8bb7-98e29eca1f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.071357 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6d34f-fe0b-4192-8bb7-98e29eca1f0e admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.072090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6d34f-fe0b-4192-8bb7-98e29eca1f0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:04:06.072401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6d34f-fe0b-4192-8bb7-98e29eca1f0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.074096 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6d34f-fe0b-4192-8bb7-98e29eca1f0e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.092617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 791/1568] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:04:06.093926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6d34f-fe0b-4192-8bb7-98e29eca1f0e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.109505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6d34f-fe0b-4192-8bb7-98e29eca1f0e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.117452 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf555e5b-b2db-4c6b-afc8-791c2858827c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.118635 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf555e5b-b2db-4c6b-afc8-791c2858827c admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.119113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf555e5b-b2db-4c6b-afc8-791c2858827c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.119495 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf555e5b-b2db-4c6b-afc8-791c2858827c admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.120520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf555e5b-b2db-4c6b-afc8-791c2858827c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.122995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf555e5b-b2db-4c6b-afc8-791c2858827c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.246576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 778/1569] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:06 2022] POST /identity/v3/projects => generated 373 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:06.246576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa6af1-790f-40a5-bf0e-8cdcfaef35c6 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.302712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 792/1570] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:06 2022] POST /identity/v3/projects => generated 365 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:06.302712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf555e5b-b2db-4c6b-afc8-791c2858827c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.340934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa6af1-790f-40a5-bf0e-8cdcfaef35c6 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.375636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf555e5b-b2db-4c6b-afc8-791c2858827c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.382041 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbc12a-ed81-4cd8-bb5b-2ae1425eaa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.390281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbc12a-ed81-4cd8-bb5b-2ae1425eaa5c admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.390281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbc12a-ed81-4cd8-bb5b-2ae1425eaa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.390281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbc12a-ed81-4cd8-bb5b-2ae1425eaa5c admin admin] PATH_INFO: `/v3/projects/5e929654eb2b4c1ca969174e94aa8de2/users/592de777b94148b791e18bf898386f34/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.405307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbc12a-ed81-4cd8-bb5b-2ae1425eaa5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e929654eb2b4c1ca969174e94aa8de2, user_id=592de777b94148b791e18bf898386f34, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.408890 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbc12a-ed81-4cd8-bb5b-2ae1425eaa5c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.421442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025482aa-078f-4f66-8636-cc3b01e2e819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.422215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025482aa-078f-4f66-8636-cc3b01e2e819 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.429437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025482aa-078f-4f66-8636-cc3b01e2e819 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.429437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025482aa-078f-4f66-8636-cc3b01e2e819 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.429437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025482aa-078f-4f66-8636-cc3b01e2e819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.429437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025482aa-078f-4f66-8636-cc3b01e2e819 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.437375 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 779/1571] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:06.438266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025482aa-078f-4f66-8636-cc3b01e2e819 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.450777 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025482aa-078f-4f66-8636-cc3b01e2e819 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.455939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d3fff-3c35-4122-bfc5-4e4e5d30fd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.456542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d3fff-3c35-4122-bfc5-4e4e5d30fd5b admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.456828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d3fff-3c35-4122-bfc5-4e4e5d30fd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.457125 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d3fff-3c35-4122-bfc5-4e4e5d30fd5b admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.457844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d3fff-3c35-4122-bfc5-4e4e5d30fd5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:04:06.458155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d3fff-3c35-4122-bfc5-4e4e5d30fd5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.459891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d3fff-3c35-4122-bfc5-4e4e5d30fd5b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.472785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 780/1572] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:04:06.473633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d3fff-3c35-4122-bfc5-4e4e5d30fd5b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.485593 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d3fff-3c35-4122-bfc5-4e4e5d30fd5b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.491572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b4c10-b828-4246-a2a7-87f2a3aec5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.492251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b4c10-b828-4246-a2a7-87f2a3aec5bf admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.492545 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b4c10-b828-4246-a2a7-87f2a3aec5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.492886 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b4c10-b828-4246-a2a7-87f2a3aec5bf admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.493600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b4c10-b828-4246-a2a7-87f2a3aec5bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.495622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b4c10-b828-4246-a2a7-87f2a3aec5bf admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.503118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 793/1573] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:06 2022] PUT /identity/v3/projects/5e929654eb2b4c1ca969174e94aa8de2/users/592de777b94148b791e18bf898386f34/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:06.503118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbc12a-ed81-4cd8-bb5b-2ae1425eaa5c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.570210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbc12a-ed81-4cd8-bb5b-2ae1425eaa5c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.575567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf45f8-6a3d-4c5d-9260-6590ce44a223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.576282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf45f8-6a3d-4c5d-9260-6590ce44a223 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.576658 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf45f8-6a3d-4c5d-9260-6590ce44a223 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.576855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf45f8-6a3d-4c5d-9260-6590ce44a223 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.577569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf45f8-6a3d-4c5d-9260-6590ce44a223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.580030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf45f8-6a3d-4c5d-9260-6590ce44a223 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.622241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:06.629519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 781/1574] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:06 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:06.631114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b4c10-b828-4246-a2a7-87f2a3aec5bf admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.700287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b4c10-b828-4246-a2a7-87f2a3aec5bf admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.705925 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945203b0-735e-4426-9803-7ab51d7e2943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.706524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945203b0-735e-4426-9803-7ab51d7e2943 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.706852 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945203b0-735e-4426-9803-7ab51d7e2943 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.707090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945203b0-735e-4426-9803-7ab51d7e2943 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.707794 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945203b0-735e-4426-9803-7ab51d7e2943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.709715 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945203b0-735e-4426-9803-7ab51d7e2943 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.725660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 794/1575] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:06 2022] POST /identity/v3/projects => generated 377 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:06.726989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf45f8-6a3d-4c5d-9260-6590ce44a223 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.755326 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:06.805151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf45f8-6a3d-4c5d-9260-6590ce44a223 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.813950 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3662c0a-42a1-4c38-acec-398e52e4d93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.815041 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3662c0a-42a1-4c38-acec-398e52e4d93f admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.815516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3662c0a-42a1-4c38-acec-398e52e4d93f admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.815802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 782/1576] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:06 2022] POST /identity/v3/projects => generated 381 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:06.816545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3662c0a-42a1-4c38-acec-398e52e4d93f admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.817248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3662c0a-42a1-4c38-acec-398e52e4d93f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.817436 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945203b0-735e-4426-9803-7ab51d7e2943 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.820966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3662c0a-42a1-4c38-acec-398e52e4d93f admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.908156 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945203b0-735e-4426-9803-7ab51d7e2943 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.914215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e1b65-d040-4bfb-a6ba-88f6140a3cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.914951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e1b65-d040-4bfb-a6ba-88f6140a3cce admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.915238 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e1b65-d040-4bfb-a6ba-88f6140a3cce admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.915597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e1b65-d040-4bfb-a6ba-88f6140a3cce admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.916285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e1b65-d040-4bfb-a6ba-88f6140a3cce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.918307 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e1b65-d040-4bfb-a6ba-88f6140a3cce admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.919955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 795/1577] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:06 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:06.921399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3662c0a-42a1-4c38-acec-398e52e4d93f admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.935704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3662c0a-42a1-4c38-acec-398e52e4d93f admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.941115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c004ebd-1aa3-41e3-aa44-413b5c0182c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.941848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c004ebd-1aa3-41e3-aa44-413b5c0182c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.942147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c004ebd-1aa3-41e3-aa44-413b5c0182c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.942457 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c004ebd-1aa3-41e3-aa44-413b5c0182c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.943195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c004ebd-1aa3-41e3-aa44-413b5c0182c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.944853 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c004ebd-1aa3-41e3-aa44-413b5c0182c4 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:06.956804 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 796/1578] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:06.957685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c004ebd-1aa3-41e3-aa44-413b5c0182c4 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:06.971747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c004ebd-1aa3-41e3-aa44-413b5c0182c4 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:06.976897 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbb9ac-f27f-4296-90f3-8dc25e0a1adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:06.977583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbb9ac-f27f-4296-90f3-8dc25e0a1adc admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:06.977868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbb9ac-f27f-4296-90f3-8dc25e0a1adc admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:06.978160 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbb9ac-f27f-4296-90f3-8dc25e0a1adc admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:06.978973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbb9ac-f27f-4296-90f3-8dc25e0a1adc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:06.980872 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbb9ac-f27f-4296-90f3-8dc25e0a1adc admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.045004 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 783/1579] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:06 2022] POST /identity/v3/users => generated 283 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:07.046675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a6421-aa31-4d26-bcfb-1c91fc8d165b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.046934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a6421-aa31-4d26-bcfb-1c91fc8d165b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.047210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a6421-aa31-4d26-bcfb-1c91fc8d165b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.069821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ce7a6421-aa31-4d26-bcfb-1c91fc8d165b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:07.071264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ce7a6421-aa31-4d26-bcfb-1c91fc8d165b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:07.080200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 784/1580] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:07.081081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a6421-aa31-4d26-bcfb-1c91fc8d165b None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.092579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a6421-aa31-4d26-bcfb-1c91fc8d165b None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.098513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d352d-72c6-46e3-835a-da4fd330c9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.099271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d352d-72c6-46e3-835a-da4fd330c9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.106711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d352d-72c6-46e3-835a-da4fd330c9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.106711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d352d-72c6-46e3-835a-da4fd330c9b3 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.106711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d352d-72c6-46e3-835a-da4fd330c9b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.106711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d352d-72c6-46e3-835a-da4fd330c9b3 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.162587 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 797/1581] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:06 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:07.163863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbb9ac-f27f-4296-90f3-8dc25e0a1adc admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.243731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbb9ac-f27f-4296-90f3-8dc25e0a1adc admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.249700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17734421-4bbc-40b5-9fcf-2ff55699acc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.250513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17734421-4bbc-40b5-9fcf-2ff55699acc4 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.250794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17734421-4bbc-40b5-9fcf-2ff55699acc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.251148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17734421-4bbc-40b5-9fcf-2ff55699acc4 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.251915 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17734421-4bbc-40b5-9fcf-2ff55699acc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.254285 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17734421-4bbc-40b5-9fcf-2ff55699acc4 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.318426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 785/1582] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:07 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:07.319591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d352d-72c6-46e3-835a-da4fd330c9b3 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.331379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d352d-72c6-46e3-835a-da4fd330c9b3 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.336818 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65ab07-42f9-4a45-b635-6adeab883e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.337541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65ab07-42f9-4a45-b635-6adeab883e9d admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.337911 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65ab07-42f9-4a45-b635-6adeab883e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.338163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65ab07-42f9-4a45-b635-6adeab883e9d admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.338847 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65ab07-42f9-4a45-b635-6adeab883e9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.340833 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65ab07-42f9-4a45-b635-6adeab883e9d admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.445547 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 798/1583] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:07 2022] POST /identity/v3/users => generated 289 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:07.446508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17734421-4bbc-40b5-9fcf-2ff55699acc4 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.458618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17734421-4bbc-40b5-9fcf-2ff55699acc4 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.464202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c250641-1cac-4ec9-9dd2-76ee71db60e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.464914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c250641-1cac-4ec9-9dd2-76ee71db60e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.465213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c250641-1cac-4ec9-9dd2-76ee71db60e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.465473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c250641-1cac-4ec9-9dd2-76ee71db60e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.466277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c250641-1cac-4ec9-9dd2-76ee71db60e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.468025 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c250641-1cac-4ec9-9dd2-76ee71db60e0 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.475034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 786/1584] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:07 2022] POST /identity/v3/users => generated 291 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:07.475034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65ab07-42f9-4a45-b635-6adeab883e9d admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.483771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 799/1585] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:07.483771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c250641-1cac-4ec9-9dd2-76ee71db60e0 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.487681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65ab07-42f9-4a45-b635-6adeab883e9d admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.494483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c250641-1cac-4ec9-9dd2-76ee71db60e0 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.496483 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0b77c-192d-446e-b5fc-3d0a872ad7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.497997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0b77c-192d-446e-b5fc-3d0a872ad7d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.498550 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0b77c-192d-446e-b5fc-3d0a872ad7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.499017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0b77c-192d-446e-b5fc-3d0a872ad7d8 admin admin] PATH_INFO: `/v3/projects/08c6be8edb884214bae7a0abbf97dc16/users/7e3f168283ba4f159948830f396ae7cf/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.500586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c0de4-50e3-43c0-8816-d9652270d14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.501984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c0de4-50e3-43c0-8816-d9652270d14f admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.502283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c0de4-50e3-43c0-8816-d9652270d14f admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.502674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c0de4-50e3-43c0-8816-d9652270d14f admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.503431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c0de4-50e3-43c0-8816-d9652270d14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.505649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c0de4-50e3-43c0-8816-d9652270d14f admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.525487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f0b77c-192d-446e-b5fc-3d0a872ad7d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c6be8edb884214bae7a0abbf97dc16, user_id=7e3f168283ba4f159948830f396ae7cf, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.526103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 800/1586] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:07.527228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: WARNING py.warnings [None req-43f0b77c-192d-446e-b5fc-3d0a872ad7d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:04:07.527228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: warnings.warn(msg) Jul 25 19:04:07.527228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: Jul 25 19:04:07.527545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c0de4-50e3-43c0-8816-d9652270d14f admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.530029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9d6c0de4-50e3-43c0-8816-d9652270d14f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:07.530789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f0b77c-192d-446e-b5fc-3d0a872ad7d8 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.620594 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:07.627213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c0de4-50e3-43c0-8816-d9652270d14f admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.631627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d2bf6-0f26-4cb5-be9b-2e3b74744ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.632550 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d2bf6-0f26-4cb5-be9b-2e3b74744ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.633031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d2bf6-0f26-4cb5-be9b-2e3b74744ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.633324 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d2bf6-0f26-4cb5-be9b-2e3b74744ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.646422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d2bf6-0f26-4cb5-be9b-2e3b74744ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.647187 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 787/1587] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:07 2022] PUT /identity/v3/projects/08c6be8edb884214bae7a0abbf97dc16/users/7e3f168283ba4f159948830f396ae7cf/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:07.648050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0b77c-192d-446e-b5fc-3d0a872ad7d8 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.651806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d2bf6-0f26-4cb5-be9b-2e3b74744ae0 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.660296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:07.666462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 801/1588] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:07.667605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d2bf6-0f26-4cb5-be9b-2e3b74744ae0 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.684602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d2bf6-0f26-4cb5-be9b-2e3b74744ae0 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.743731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0b77c-192d-446e-b5fc-3d0a872ad7d8 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.749862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244736a-5438-4450-aac1-1b9455370f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.750617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244736a-5438-4450-aac1-1b9455370f25 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.750900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244736a-5438-4450-aac1-1b9455370f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.751185 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244736a-5438-4450-aac1-1b9455370f25 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.752654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244736a-5438-4450-aac1-1b9455370f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.754792 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244736a-5438-4450-aac1-1b9455370f25 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.770227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86d030-93c8-4ce3-af95-ff9595cfdb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.771005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86d030-93c8-4ce3-af95-ff9595cfdb8c admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.771464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86d030-93c8-4ce3-af95-ff9595cfdb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.771768 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86d030-93c8-4ce3-af95-ff9595cfdb8c admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.772581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86d030-93c8-4ce3-af95-ff9595cfdb8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.775034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86d030-93c8-4ce3-af95-ff9595cfdb8c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.788889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 802/1589] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:07.789938 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86d030-93c8-4ce3-af95-ff9595cfdb8c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.802684 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86d030-93c8-4ce3-af95-ff9595cfdb8c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.808381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06585705-e19c-43f3-8893-4e6bcdd8c6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.809187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06585705-e19c-43f3-8893-4e6bcdd8c6ed admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.809774 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06585705-e19c-43f3-8893-4e6bcdd8c6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.810065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06585705-e19c-43f3-8893-4e6bcdd8c6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.810947 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06585705-e19c-43f3-8893-4e6bcdd8c6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.812618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06585705-e19c-43f3-8893-4e6bcdd8c6ed admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.825937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 803/1590] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:07.827093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06585705-e19c-43f3-8893-4e6bcdd8c6ed admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.838428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06585705-e19c-43f3-8893-4e6bcdd8c6ed admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.843950 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9c851-bb90-4bf7-aa2b-82d7e037fd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.844647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9c851-bb90-4bf7-aa2b-82d7e037fd67 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.844937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9c851-bb90-4bf7-aa2b-82d7e037fd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.845337 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9c851-bb90-4bf7-aa2b-82d7e037fd67 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.846167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9c851-bb90-4bf7-aa2b-82d7e037fd67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.847864 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9c851-bb90-4bf7-aa2b-82d7e037fd67 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.860020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 804/1591] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:07.861642 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9c851-bb90-4bf7-aa2b-82d7e037fd67 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.877118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9c851-bb90-4bf7-aa2b-82d7e037fd67 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.886386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12e112-14e0-4700-829d-78c172de623e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.887960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12e112-14e0-4700-829d-78c172de623e admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.888920 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12e112-14e0-4700-829d-78c172de623e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.889088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12e112-14e0-4700-829d-78c172de623e admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.890491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12e112-14e0-4700-829d-78c172de623e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.893766 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12e112-14e0-4700-829d-78c172de623e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.907072 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 805/1592] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:07.908014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12e112-14e0-4700-829d-78c172de623e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.921721 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12e112-14e0-4700-829d-78c172de623e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.930501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554afd2-3deb-49ee-b798-5d99fdac96e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.931236 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554afd2-3deb-49ee-b798-5d99fdac96e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.931528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554afd2-3deb-49ee-b798-5d99fdac96e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.931812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554afd2-3deb-49ee-b798-5d99fdac96e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.932562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554afd2-3deb-49ee-b798-5d99fdac96e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.934388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554afd2-3deb-49ee-b798-5d99fdac96e3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.947034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 806/1593] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:07.952105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88417be3-6a86-4c33-bf0f-9d064b157e16 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.957477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88417be3-6a86-4c33-bf0f-9d064b157e16 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.957477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88417be3-6a86-4c33-bf0f-9d064b157e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.957778 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 788/1594] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:07 2022] POST /identity/v3/users => generated 287 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:07.958225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244736a-5438-4450-aac1-1b9455370f25 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:07.972918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244736a-5438-4450-aac1-1b9455370f25 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:07.979776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d5cab-8fc9-495b-96d2-e41283e13feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:07.980602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d5cab-8fc9-495b-96d2-e41283e13feb admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:07.980996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d5cab-8fc9-495b-96d2-e41283e13feb admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:07.981293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d5cab-8fc9-495b-96d2-e41283e13feb admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:07.982124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d5cab-8fc9-495b-96d2-e41283e13feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:07.982401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-88417be3-6a86-4c33-bf0f-9d064b157e16 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:07.983798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d5cab-8fc9-495b-96d2-e41283e13feb admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:07.984472 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-88417be3-6a86-4c33-bf0f-9d064b157e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:07.998508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 807/1595] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:07.998508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88417be3-6a86-4c33-bf0f-9d064b157e16 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:08.003863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 789/1596] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:08.004685 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d5cab-8fc9-495b-96d2-e41283e13feb admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:08.016454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88417be3-6a86-4c33-bf0f-9d064b157e16 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:08.022248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d5cab-8fc9-495b-96d2-e41283e13feb admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:08.025805 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1895c-74ea-4d01-9238-253e3e441a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:08.027051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1895c-74ea-4d01-9238-253e3e441a4b admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:08.027558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1895c-74ea-4d01-9238-253e3e441a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:08.027980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1895c-74ea-4d01-9238-253e3e441a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:08.029296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1895c-74ea-4d01-9238-253e3e441a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:08.029721 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4bfa5-3388-4074-873a-c761f8d91fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:08.030738 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4bfa5-3388-4074-873a-c761f8d91fef admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:08.031176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4bfa5-3388-4074-873a-c761f8d91fef admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:08.031650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4bfa5-3388-4074-873a-c761f8d91fef admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:08.032406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1895c-74ea-4d01-9238-253e3e441a4b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:08.033454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4bfa5-3388-4074-873a-c761f8d91fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:08.036339 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4bfa5-3388-4074-873a-c761f8d91fef admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:08.057194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 808/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:08.057194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1895c-74ea-4d01-9238-253e3e441a4b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:08.057526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 790/1598] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:08.059332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4bfa5-3388-4074-873a-c761f8d91fef admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:08.076627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4bfa5-3388-4074-873a-c761f8d91fef admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:08.079602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1895c-74ea-4d01-9238-253e3e441a4b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:08.088251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd48145-7a21-4c8a-aefc-714bdb0509f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:08.089722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd48145-7a21-4c8a-aefc-714bdb0509f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:08.089946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5ce41-bd57-425a-8491-8713b26885fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:08.090129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd48145-7a21-4c8a-aefc-714bdb0509f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:08.090369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd48145-7a21-4c8a-aefc-714bdb0509f6 admin admin] PATH_INFO: `/v3/projects/e02ab65920c04a8296db23c2ba37bd14/users/88f57dc362e746f98cea2b84e8b1e351/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:08.090947 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5ce41-bd57-425a-8491-8713b26885fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:08.091252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5ce41-bd57-425a-8491-8713b26885fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:08.091535 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5ce41-bd57-425a-8491-8713b26885fe admin admin] PATH_INFO: `/v3/projects/0645a63ae797444585edd4eb2924f1e6/users/a324e94f4e1b4199b684514b8818dda4/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:08.114542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5ce41-bd57-425a-8491-8713b26885fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0645a63ae797444585edd4eb2924f1e6, user_id=a324e94f4e1b4199b684514b8818dda4, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:08.117584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5ce41-bd57-425a-8491-8713b26885fe admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:08.118978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd48145-7a21-4c8a-aefc-714bdb0509f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02ab65920c04a8296db23c2ba37bd14, user_id=88f57dc362e746f98cea2b84e8b1e351, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:08.124303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd48145-7a21-4c8a-aefc-714bdb0509f6 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:08.202876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 809/1599] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:08 2022] PUT /identity/v3/projects/0645a63ae797444585edd4eb2924f1e6/users/a324e94f4e1b4199b684514b8818dda4/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:08.204810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5ce41-bd57-425a-8491-8713b26885fe admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:08.243119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 791/1600] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:08 2022] PUT /identity/v3/projects/e02ab65920c04a8296db23c2ba37bd14/users/88f57dc362e746f98cea2b84e8b1e351/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:08.245304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd48145-7a21-4c8a-aefc-714bdb0509f6 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:08.321141 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5ce41-bd57-425a-8491-8713b26885fe admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:08.326138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:08.352349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd48145-7a21-4c8a-aefc-714bdb0509f6 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:08.359292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f7103-de86-4a55-a068-3f3b9882ac50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:08.360396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f7103-de86-4a55-a068-3f3b9882ac50 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:08.361313 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f7103-de86-4a55-a068-3f3b9882ac50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:08.361452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f7103-de86-4a55-a068-3f3b9882ac50 admin admin] PATH_INFO: `/v3/projects/c2de35c4252b4a6fa55eb7e5d59e06cf/users/c99e02a845184f0aa206ba9f209ae4ef/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:08.391388 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f7103-de86-4a55-a068-3f3b9882ac50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2de35c4252b4a6fa55eb7e5d59e06cf, user_id=c99e02a845184f0aa206ba9f209ae4ef, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:08.398145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f7103-de86-4a55-a068-3f3b9882ac50 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:08.411851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c021060-0030-4ed7-a625-a273565cedf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:08.413203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c021060-0030-4ed7-a625-a273565cedf4 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:08.413203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c021060-0030-4ed7-a625-a273565cedf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:08.413203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c021060-0030-4ed7-a625-a273565cedf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:08.414145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c021060-0030-4ed7-a625-a273565cedf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:08.416606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c021060-0030-4ed7-a625-a273565cedf4 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:08.435957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 810/1601] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:08.437566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c021060-0030-4ed7-a625-a273565cedf4 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:08.526243 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c021060-0030-4ed7-a625-a273565cedf4 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:08.527295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 792/1602] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:08 2022] PUT /identity/v3/projects/c2de35c4252b4a6fa55eb7e5d59e06cf/users/c99e02a845184f0aa206ba9f209ae4ef/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:08.528292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f7103-de86-4a55-a068-3f3b9882ac50 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:08.584616 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:08.604008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5957b91-0fb8-4bbc-b0e9-f4592bd062f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:08.604852 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5957b91-0fb8-4bbc-b0e9-f4592bd062f7 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:08.606939 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5957b91-0fb8-4bbc-b0e9-f4592bd062f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:08.606939 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5957b91-0fb8-4bbc-b0e9-f4592bd062f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:08.625464 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f7103-de86-4a55-a068-3f3b9882ac50 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:08.636574 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063591e-7bd1-4fd9-af8f-b188d3ae872d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:08.638397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063591e-7bd1-4fd9-af8f-b188d3ae872d admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:08.638722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063591e-7bd1-4fd9-af8f-b188d3ae872d admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:08.639011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063591e-7bd1-4fd9-af8f-b188d3ae872d admin admin] PATH_INFO: `/v3/projects/463f573db9d84c03900d443e4f6585cd/users/c0464996da4f44008ca8b91c837ac22d/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:08.660545 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063591e-7bd1-4fd9-af8f-b188d3ae872d admin admin] RBAC: Authorizing `identity:create_grant(project_id=463f573db9d84c03900d443e4f6585cd, user_id=c0464996da4f44008ca8b91c837ac22d, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:08.664300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063591e-7bd1-4fd9-af8f-b188d3ae872d admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:08.702169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5957b91-0fb8-4bbc-b0e9-f4592bd062f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:08.705706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5957b91-0fb8-4bbc-b0e9-f4592bd062f7 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:08.721765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 811/1603] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:08.722644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5957b91-0fb8-4bbc-b0e9-f4592bd062f7 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:08.732986 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5957b91-0fb8-4bbc-b0e9-f4592bd062f7 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:08.738245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498a006-2a9c-4dc0-81dc-d8802c21ed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:08.739774 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498a006-2a9c-4dc0-81dc-d8802c21ed33 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:08.740058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498a006-2a9c-4dc0-81dc-d8802c21ed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:08.740435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498a006-2a9c-4dc0-81dc-d8802c21ed33 admin admin] PATH_INFO: `/v3/projects/31a9679793634657a266513b88ea7d88/users/9481d8c7a5a743aa8ad0e66d9a718cc4/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:08.744665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:08.759508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2498a006-2a9c-4dc0-81dc-d8802c21ed33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a9679793634657a266513b88ea7d88, user_id=9481d8c7a5a743aa8ad0e66d9a718cc4, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:08.762378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2498a006-2a9c-4dc0-81dc-d8802c21ed33 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:08.794499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 793/1604] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:08 2022] PUT /identity/v3/projects/463f573db9d84c03900d443e4f6585cd/users/c0464996da4f44008ca8b91c837ac22d/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:08.800443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b3e46-0387-4764-937d-a33d52bb1deb None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:08.800443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b3e46-0387-4764-937d-a33d52bb1deb None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:08.800443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b3e46-0387-4764-937d-a33d52bb1deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:08.833835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ca5b3e46-0387-4764-937d-a33d52bb1deb None None] MFA Rules not processed for user `a324e94f4e1b4199b684514b8818dda4`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:08.835139 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ca5b3e46-0387-4764-937d-a33d52bb1deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:08.921478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 794/1605] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:08.923946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb34d7-54ef-407c-80e5-5dc8bb0ab17a None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:08.924445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb34d7-54ef-407c-80e5-5dc8bb0ab17a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:08.924953 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb34d7-54ef-407c-80e5-5dc8bb0ab17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:08.958748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 812/1606] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:08 2022] PUT /identity/v3/projects/31a9679793634657a266513b88ea7d88/users/9481d8c7a5a743aa8ad0e66d9a718cc4/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:08.958748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498a006-2a9c-4dc0-81dc-d8802c21ed33 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:08.974899 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-68eb34d7-54ef-407c-80e5-5dc8bb0ab17a None None] MFA Rules not processed for user `88f57dc362e746f98cea2b84e8b1e351`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:08.977253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-68eb34d7-54ef-407c-80e5-5dc8bb0ab17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:09.092966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498a006-2a9c-4dc0-81dc-d8802c21ed33 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:09.099810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbd4c8-4307-482e-a546-12f021b3afa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:09.100664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbd4c8-4307-482e-a546-12f021b3afa6 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:09.100969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbd4c8-4307-482e-a546-12f021b3afa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:09.101469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbd4c8-4307-482e-a546-12f021b3afa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:09.102290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbd4c8-4307-482e-a546-12f021b3afa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:09.104322 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 795/1607] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:09.105588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57ff4f-4a51-41e9-b978-a44b9c56a287 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:09.106003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57ff4f-4a51-41e9-b978-a44b9c56a287 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:09.106303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57ff4f-4a51-41e9-b978-a44b9c56a287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:09.111536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbd4c8-4307-482e-a546-12f021b3afa6 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:09.141413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 813/1608] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:09.142660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dca93-23df-4900-a362-d0dde8630af5 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:09.149148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dca93-23df-4900-a362-d0dde8630af5 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:09.149148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dca93-23df-4900-a362-d0dde8630af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:09.150916 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-1d57ff4f-4a51-41e9-b978-a44b9c56a287 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:09.152957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1d57ff4f-4a51-41e9-b978-a44b9c56a287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:09.169463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 796/1609] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:09.171108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1454e57-328d-43a5-95d6-b1bf937f73a4 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:09.171588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1454e57-328d-43a5-95d6-b1bf937f73a4 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:09.171785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1454e57-328d-43a5-95d6-b1bf937f73a4 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:09.172862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-769dca93-23df-4900-a362-d0dde8630af5 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:09.175785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 797/1610] 10.222.0.17 () {54 vars in 962 bytes} [Mon Jul 25 19:04:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:04:09.177341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af241817-f1d7-4225-ae85-fde2beb87ee2 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:09.177842 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af241817-f1d7-4225-ae85-fde2beb87ee2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:09.178518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af241817-f1d7-4225-ae85-fde2beb87ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:09.184104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-769dca93-23df-4900-a362-d0dde8630af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:09.203782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 814/1611] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:09.203782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dca93-23df-4900-a362-d0dde8630af5 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:09.229798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-af241817-f1d7-4225-ae85-fde2beb87ee2 None None] MFA Rules not processed for user `9481d8c7a5a743aa8ad0e66d9a718cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:09.231829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-af241817-f1d7-4225-ae85-fde2beb87ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:09.267329 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:09.301243 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dca93-23df-4900-a362-d0dde8630af5 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:09.306863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8de276-64eb-4afa-9b90-0177dd580157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:09.330258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8de276-64eb-4afa-9b90-0177dd580157 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:09.330258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8de276-64eb-4afa-9b90-0177dd580157 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:09.330258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8de276-64eb-4afa-9b90-0177dd580157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:09.330258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8de276-64eb-4afa-9b90-0177dd580157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:09.330258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8de276-64eb-4afa-9b90-0177dd580157 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:09.338989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 815/1612] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:09.339911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8de276-64eb-4afa-9b90-0177dd580157 service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:09.343346 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 798/1613] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:09.345109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31188812-4eca-4b0b-8c57-f0fca9b7bf85 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:09.345276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31188812-4eca-4b0b-8c57-f0fca9b7bf85 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:09.345840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31188812-4eca-4b0b-8c57-f0fca9b7bf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:09.361880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8de276-64eb-4afa-9b90-0177dd580157 service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:09.378289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da4496-153d-4add-8bbb-7363248deb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:09.378289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da4496-153d-4add-8bbb-7363248deb3d admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:09.378289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da4496-153d-4add-8bbb-7363248deb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:09.378289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da4496-153d-4add-8bbb-7363248deb3d admin admin] PATH_INFO: `/v3/projects/1aaea95be92540759ce95f8af5c4f4b9/users/6ca991be6007459eaf7a4f172ead0f85/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:09.398990 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da4496-153d-4add-8bbb-7363248deb3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aaea95be92540759ce95f8af5c4f4b9, user_id=6ca991be6007459eaf7a4f172ead0f85, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:09.400667 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-31188812-4eca-4b0b-8c57-f0fca9b7bf85 None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:09.402094 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-31188812-4eca-4b0b-8c57-f0fca9b7bf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:09.402272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da4496-153d-4add-8bbb-7363248deb3d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:09.518401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 799/1614] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Jul 25 19:04:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:09.519516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31188812-4eca-4b0b-8c57-f0fca9b7bf85 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:09.579249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 816/1615] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:09 2022] PUT /identity/v3/projects/1aaea95be92540759ce95f8af5c4f4b9/users/6ca991be6007459eaf7a4f172ead0f85/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:09.582100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da4496-153d-4add-8bbb-7363248deb3d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:09.635009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31188812-4eca-4b0b-8c57-f0fca9b7bf85 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:09.692139 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da4496-153d-4add-8bbb-7363248deb3d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:09.697226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51615c26-fdb8-4d7c-b311-a45afacf9558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:09.698511 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51615c26-fdb8-4d7c-b311-a45afacf9558 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:09.698861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51615c26-fdb8-4d7c-b311-a45afacf9558 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:09.703973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51615c26-fdb8-4d7c-b311-a45afacf9558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:09.711508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:09.740938 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf0bb7-c6b5-4aba-bc77-8ba6cec4cca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:09.741849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf0bb7-c6b5-4aba-bc77-8ba6cec4cca9 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:09.743243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf0bb7-c6b5-4aba-bc77-8ba6cec4cca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:09.743602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf0bb7-c6b5-4aba-bc77-8ba6cec4cca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:09.808175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51615c26-fdb8-4d7c-b311-a45afacf9558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:09.810348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51615c26-fdb8-4d7c-b311-a45afacf9558 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:09.823972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 817/1616] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:09.825072 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51615c26-fdb8-4d7c-b311-a45afacf9558 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:09.839154 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51615c26-fdb8-4d7c-b311-a45afacf9558 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:09.845319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd11687-bf74-4132-88ab-f1abe4acf652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:09.846402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd11687-bf74-4132-88ab-f1abe4acf652 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:09.846702 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd11687-bf74-4132-88ab-f1abe4acf652 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:09.847006 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd11687-bf74-4132-88ab-f1abe4acf652 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:09.847930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd11687-bf74-4132-88ab-f1abe4acf652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:09.850187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd11687-bf74-4132-88ab-f1abe4acf652 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:09.868866 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf0bb7-c6b5-4aba-bc77-8ba6cec4cca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:09.872156 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf0bb7-c6b5-4aba-bc77-8ba6cec4cca9 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:09.906088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:09.928623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 800/1617] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:09.928725 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf0bb7-c6b5-4aba-bc77-8ba6cec4cca9 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:10.055471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf0bb7-c6b5-4aba-bc77-8ba6cec4cca9 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:10.062965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cc4b0-f50d-4353-b0bd-5cd897d53016 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:10.063806 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cc4b0-f50d-4353-b0bd-5cd897d53016 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:10.064856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cc4b0-f50d-4353-b0bd-5cd897d53016 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:10.065201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cc4b0-f50d-4353-b0bd-5cd897d53016 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:10.105234 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 818/1618] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:09 2022] POST /identity/v3/projects => generated 381 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:10.106773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762a02e-7beb-425a-bb29-e38af3ec2c61 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:10.107080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762a02e-7beb-425a-bb29-e38af3ec2c61 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:10.107366 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762a02e-7beb-425a-bb29-e38af3ec2c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:10.145219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-9762a02e-7beb-425a-bb29-e38af3ec2c61 None None] MFA Rules not processed for user `6ca991be6007459eaf7a4f172ead0f85`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:10.146575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9762a02e-7beb-425a-bb29-e38af3ec2c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:10.150133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cc4b0-f50d-4353-b0bd-5cd897d53016 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:10.152616 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cc4b0-f50d-4353-b0bd-5cd897d53016 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:10.169042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 801/1619] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:10.171168 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cc4b0-f50d-4353-b0bd-5cd897d53016 service nova] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:10.199688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cc4b0-f50d-4353-b0bd-5cd897d53016 service nova] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:10.341769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:10.352156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 819/1620] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:10.357004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762a02e-7beb-425a-bb29-e38af3ec2c61 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:10.381993 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4768b32-000c-4232-ab44-8d03cd4fecb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:10.383678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4768b32-000c-4232-ab44-8d03cd4fecb3 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:10.384428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4768b32-000c-4232-ab44-8d03cd4fecb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:10.385351 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4768b32-000c-4232-ab44-8d03cd4fecb3 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:10.388715 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4768b32-000c-4232-ab44-8d03cd4fecb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:10.394434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4768b32-000c-4232-ab44-8d03cd4fecb3 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:10.484597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762a02e-7beb-425a-bb29-e38af3ec2c61 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:10.491840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3603baf-63d9-4772-a7fa-ea156d90d5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:10.492705 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3603baf-63d9-4772-a7fa-ea156d90d5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:10.493015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3603baf-63d9-4772-a7fa-ea156d90d5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:10.493599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3603baf-63d9-4772-a7fa-ea156d90d5f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:10.494426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3603baf-63d9-4772-a7fa-ea156d90d5f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:10.496611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3603baf-63d9-4772-a7fa-ea156d90d5f2 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:10.603075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 802/1621] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:10 2022] POST /identity/v3/users => generated 297 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:10.606747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4768b32-000c-4232-ab44-8d03cd4fecb3 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:10.629149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:10.637924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4768b32-000c-4232-ab44-8d03cd4fecb3 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:10.650598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 820/1622] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:10 2022] POST /identity/v3/projects => generated 371 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:04:10.652291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3603baf-63d9-4772-a7fa-ea156d90d5f2 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:10.653994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a78de8-9000-4ef2-b1b6-9491d2fc8013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:10.654769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a78de8-9000-4ef2-b1b6-9491d2fc8013 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:10.655061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a78de8-9000-4ef2-b1b6-9491d2fc8013 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:10.655339 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a78de8-9000-4ef2-b1b6-9491d2fc8013 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:10.656263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a78de8-9000-4ef2-b1b6-9491d2fc8013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:10.658563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a78de8-9000-4ef2-b1b6-9491d2fc8013 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:10.696884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 803/1623] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:04:10.696884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a78de8-9000-4ef2-b1b6-9491d2fc8013 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:10.720624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a78de8-9000-4ef2-b1b6-9491d2fc8013 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:10.745967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3603baf-63d9-4772-a7fa-ea156d90d5f2 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:10.752111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7a117-490e-4c66-8f52-bc181dc96836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:10.753083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7a117-490e-4c66-8f52-bc181dc96836 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:10.753376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7a117-490e-4c66-8f52-bc181dc96836 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:10.753660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7a117-490e-4c66-8f52-bc181dc96836 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:10.754485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7a117-490e-4c66-8f52-bc181dc96836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:10.756840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7a117-490e-4c66-8f52-bc181dc96836 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:10.854904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0e02f-49b6-4d93-89bc-b48e4c5d0cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:10.856309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0e02f-49b6-4d93-89bc-b48e4c5d0cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:10.856784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0e02f-49b6-4d93-89bc-b48e4c5d0cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:10.857265 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0e02f-49b6-4d93-89bc-b48e4c5d0cbb admin admin] PATH_INFO: `/v3/projects/5cee2cf82c01477db791e979a1a8e3ac/users/9ee4896bed48485fa71517a20853311e/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:10.885988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0e02f-49b6-4d93-89bc-b48e4c5d0cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cee2cf82c01477db791e979a1a8e3ac, user_id=9ee4896bed48485fa71517a20853311e, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:10.889150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0e02f-49b6-4d93-89bc-b48e4c5d0cbb admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:11.026765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 821/1624] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:10 2022] POST /identity/v3/users => generated 292 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:11.028830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7a117-490e-4c66-8f52-bc181dc96836 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:11.054119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7a117-490e-4c66-8f52-bc181dc96836 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:11.072195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835f65b-97bb-4111-8960-61dbd3deae31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:11.072195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835f65b-97bb-4111-8960-61dbd3deae31 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:11.072195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835f65b-97bb-4111-8960-61dbd3deae31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:11.072195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835f65b-97bb-4111-8960-61dbd3deae31 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:11.075608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835f65b-97bb-4111-8960-61dbd3deae31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:11.075608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835f65b-97bb-4111-8960-61dbd3deae31 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:11.096633 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 822/1625] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:11.097895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835f65b-97bb-4111-8960-61dbd3deae31 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:11.110517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835f65b-97bb-4111-8960-61dbd3deae31 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:11.145241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 804/1626] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:10 2022] PUT /identity/v3/projects/5cee2cf82c01477db791e979a1a8e3ac/users/9ee4896bed48485fa71517a20853311e/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:11.145241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88dff3-ae82-4968-9666-9a30dc3c9127 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:11.145241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88dff3-ae82-4968-9666-9a30dc3c9127 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:11.145241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88dff3-ae82-4968-9666-9a30dc3c9127 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:11.666556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876171b-d94d-420b-9e47-1d2086763bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:11.666556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876171b-d94d-420b-9e47-1d2086763bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:11.666556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876171b-d94d-420b-9e47-1d2086763bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:11.666556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876171b-d94d-420b-9e47-1d2086763bd0 admin admin] PATH_INFO: `/v3/projects/21ca773a37074b7987cb0d679e6ad13f/users/5ad8619858734853825fafa1ba6cd021/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:11.669716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 805/1627] 10.222.0.17 () {54 vars in 962 bytes} [Mon Jul 25 19:04:11 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:04:11.669716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88dff3-ae82-4968-9666-9a30dc3c9127 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:11.671120 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876171b-d94d-420b-9e47-1d2086763bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ca773a37074b7987cb0d679e6ad13f, user_id=5ad8619858734853825fafa1ba6cd021, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:11.671120 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876171b-d94d-420b-9e47-1d2086763bd0 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:11.671471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88dff3-ae82-4968-9666-9a30dc3c9127 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:11.671471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d10d3-86c2-4c32-b871-a7edb3a2578a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:11.671471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d10d3-86c2-4c32-b871-a7edb3a2578a admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:11.671471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d10d3-86c2-4c32-b871-a7edb3a2578a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:11.672111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d10d3-86c2-4c32-b871-a7edb3a2578a admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:11.672111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d10d3-86c2-4c32-b871-a7edb3a2578a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:11.672111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d10d3-86c2-4c32-b871-a7edb3a2578a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:11.672111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 806/1628] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:11.672111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf4e65-cb23-4690-913b-fb5f9f50c3af None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:11.672111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf4e65-cb23-4690-913b-fb5f9f50c3af None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:11.672711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf4e65-cb23-4690-913b-fb5f9f50c3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:11.672711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-caaf4e65-cb23-4690-913b-fb5f9f50c3af None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:11.672711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-caaf4e65-cb23-4690-913b-fb5f9f50c3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:11.672711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:11.673763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 823/1629] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:11 2022] PUT /identity/v3/projects/21ca773a37074b7987cb0d679e6ad13f/users/5ad8619858734853825fafa1ba6cd021/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:04:11.673763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876171b-d94d-420b-9e47-1d2086763bd0 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:11.674030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 807/1630] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Jul 25 19:04:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:11.674030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf4e65-cb23-4690-913b-fb5f9f50c3af None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:11.674030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf4e65-cb23-4690-913b-fb5f9f50c3af None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:11.825840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db53c93b-a07a-4115-afa3-bbfdac6b7ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:11.826971 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53c93b-a07a-4115-afa3-bbfdac6b7ccf admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:11.827704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53c93b-a07a-4115-afa3-bbfdac6b7ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:11.828151 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53c93b-a07a-4115-afa3-bbfdac6b7ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:11.829629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53c93b-a07a-4115-afa3-bbfdac6b7ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:11.831595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53c93b-a07a-4115-afa3-bbfdac6b7ccf admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:11.843295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876171b-d94d-420b-9e47-1d2086763bd0 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:11.854131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e78a9d-40ce-4d5a-86b0-1a93681a0ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:11.855180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e78a9d-40ce-4d5a-86b0-1a93681a0ff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:11.855620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e78a9d-40ce-4d5a-86b0-1a93681a0ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:11.856094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e78a9d-40ce-4d5a-86b0-1a93681a0ff5 admin admin] PATH_INFO: `/v3/projects/5cee2cf82c01477db791e979a1a8e3ac/users/9ee4896bed48485fa71517a20853311e/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:11.858678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 808/1631] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:11.860600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e1cf4-0382-48b3-a7d0-17e83e8969fd None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:11.861163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e1cf4-0382-48b3-a7d0-17e83e8969fd None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:11.861550 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e1cf4-0382-48b3-a7d0-17e83e8969fd None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:11.863236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 809/1632] 10.222.0.17 () {54 vars in 912 bytes} [Mon Jul 25 19:04:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:04:11.864371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e78a9d-40ce-4d5a-86b0-1a93681a0ff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cee2cf82c01477db791e979a1a8e3ac, user_id=9ee4896bed48485fa71517a20853311e, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:11.867340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e78a9d-40ce-4d5a-86b0-1a93681a0ff5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:11.870719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e1cf4-0382-48b3-a7d0-17e83e8969fd None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:11.898680 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e1cf4-0382-48b3-a7d0-17e83e8969fd None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:11.901116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 824/1633] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:11 2022] PUT /identity/v3/projects/5cee2cf82c01477db791e979a1a8e3ac/users/9ee4896bed48485fa71517a20853311e/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:11.905202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c051f4-0cc8-4efb-adb6-c7f50188db6e None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:11.905597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c051f4-0cc8-4efb-adb6-c7f50188db6e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:11.905983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c051f4-0cc8-4efb-adb6-c7f50188db6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:11.939998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-91c051f4-0cc8-4efb-adb6-c7f50188db6e None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:11.942175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-91c051f4-0cc8-4efb-adb6-c7f50188db6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:12.018247 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb7421-24a8-45d9-8647-3ccce114679a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:12.019980 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb7421-24a8-45d9-8647-3ccce114679a admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:12.020972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb7421-24a8-45d9-8647-3ccce114679a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:12.022441 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb7421-24a8-45d9-8647-3ccce114679a admin admin] PATH_INFO: `/v3/projects/21ca773a37074b7987cb0d679e6ad13f/users/5ad8619858734853825fafa1ba6cd021/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:12.025974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb7421-24a8-45d9-8647-3ccce114679a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ca773a37074b7987cb0d679e6ad13f, user_id=5ad8619858734853825fafa1ba6cd021, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:12.030744 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb7421-24a8-45d9-8647-3ccce114679a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:12.036267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 825/1634] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:04:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:12.037747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14727d-6c97-4a62-87b5-48779b0e0c6b None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:12.038150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14727d-6c97-4a62-87b5-48779b0e0c6b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:12.038574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14727d-6c97-4a62-87b5-48779b0e0c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:12.064958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 810/1635] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:11 2022] PUT /identity/v3/projects/21ca773a37074b7987cb0d679e6ad13f/users/5ad8619858734853825fafa1ba6cd021/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:12.064958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb7421-24a8-45d9-8647-3ccce114679a admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:12.081623 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-9e14727d-6c97-4a62-87b5-48779b0e0c6b None None] MFA Rules not processed for user `9ee4896bed48485fa71517a20853311e`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:12.089048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9e14727d-6c97-4a62-87b5-48779b0e0c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:12.145574 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb7421-24a8-45d9-8647-3ccce114679a admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:12.152423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18405323-bb5e-4e78-8b41-793db4071ec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:12.153107 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18405323-bb5e-4e78-8b41-793db4071ec7 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:12.153506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18405323-bb5e-4e78-8b41-793db4071ec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:12.153798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18405323-bb5e-4e78-8b41-793db4071ec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:12.226239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 826/1636] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:12.251776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e3bcc-945a-4182-a18a-804d10bac878 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:12.251776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e3bcc-945a-4182-a18a-804d10bac878 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:12.251776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e3bcc-945a-4182-a18a-804d10bac878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:12.279995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18405323-bb5e-4e78-8b41-793db4071ec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:12.281819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18405323-bb5e-4e78-8b41-793db4071ec7 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:12.283455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-3f0e3bcc-945a-4182-a18a-804d10bac878 None None] MFA Rules not processed for user `5ad8619858734853825fafa1ba6cd021`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:12.285335 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3f0e3bcc-945a-4182-a18a-804d10bac878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:12.299904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 811/1637] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:12.300836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18405323-bb5e-4e78-8b41-793db4071ec7 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:12.320281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18405323-bb5e-4e78-8b41-793db4071ec7 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:12.324727 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e0ebc-2eaa-4dbd-aa00-f1df97dab85b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:12.326024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e0ebc-2eaa-4dbd-aa00-f1df97dab85b service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:12.326375 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e0ebc-2eaa-4dbd-aa00-f1df97dab85b service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:12.326849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e0ebc-2eaa-4dbd-aa00-f1df97dab85b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:12.358256 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e0ebc-2eaa-4dbd-aa00-f1df97dab85b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:12.375108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e0ebc-2eaa-4dbd-aa00-f1df97dab85b service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:12.410755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 812/1638] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:12.470614 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73842d46-f927-43ba-8cb3-3d3fafccce6a None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:12.471259 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73842d46-f927-43ba-8cb3-3d3fafccce6a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:12.471775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73842d46-f927-43ba-8cb3-3d3fafccce6a None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:12.475167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 813/1639] 10.222.0.17 () {54 vars in 912 bytes} [Mon Jul 25 19:04:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:04:12.476167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 827/1640] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:12.483025 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c35e2f-4f06-4aca-8c8e-26165cb2d881 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:12.483368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c35e2f-4f06-4aca-8c8e-26165cb2d881 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:12.483780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c35e2f-4f06-4aca-8c8e-26165cb2d881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:12.538108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-55c35e2f-4f06-4aca-8c8e-26165cb2d881 None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:12.538409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e3bcc-945a-4182-a18a-804d10bac878 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:12.542538 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-55c35e2f-4f06-4aca-8c8e-26165cb2d881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:12.547122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e3bcc-945a-4182-a18a-804d10bac878 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:12.552602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c85285-00f9-44ef-88ef-1df76dddf42a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:12.553428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c85285-00f9-44ef-88ef-1df76dddf42a service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:12.553866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c85285-00f9-44ef-88ef-1df76dddf42a service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:12.554441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c85285-00f9-44ef-88ef-1df76dddf42a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:12.571796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c85285-00f9-44ef-88ef-1df76dddf42a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:12.574238 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c85285-00f9-44ef-88ef-1df76dddf42a service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:12.585582 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 814/1641] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:04:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:04:12.592027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 828/1642] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:12 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:12.660845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c35e2f-4f06-4aca-8c8e-26165cb2d881 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:12.673557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c35e2f-4f06-4aca-8c8e-26165cb2d881 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:12.677688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa22e7c-c207-4b9d-ae3b-5e8a3c64fbd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:12.678753 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa22e7c-c207-4b9d-ae3b-5e8a3c64fbd4 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:12.678915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa22e7c-c207-4b9d-ae3b-5e8a3c64fbd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:12.679158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa22e7c-c207-4b9d-ae3b-5e8a3c64fbd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:12.694059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa22e7c-c207-4b9d-ae3b-5e8a3c64fbd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:12.695572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa22e7c-c207-4b9d-ae3b-5e8a3c64fbd4 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:12.710552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 815/1643] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:12.911680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c85285-00f9-44ef-88ef-1df76dddf42a service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:13.035612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c85285-00f9-44ef-88ef-1df76dddf42a service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:13.044328 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:13.046010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:13.046480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:13.047331 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] PATH_INFO: `/v3/users/a324e94f4e1b4199b684514b8818dda4` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:13.049663 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a324e94f4e1b4199b684514b8818dda4)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:13.059777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:13.071485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa22e7c-c207-4b9d-ae3b-5e8a3c64fbd4 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:13.086640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa22e7c-c207-4b9d-ae3b-5e8a3c64fbd4 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:13.096165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cbfc5-c302-40ef-9760-b65145a33e72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:13.096590 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cbfc5-c302-40ef-9760-b65145a33e72 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:13.096828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cbfc5-c302-40ef-9760-b65145a33e72 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:13.097054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cbfc5-c302-40ef-9760-b65145a33e72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:13.220171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cbfc5-c302-40ef-9760-b65145a33e72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:13.222674 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cbfc5-c302-40ef-9760-b65145a33e72 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:13.240736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 816/1644] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:13.244336 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cbfc5-c302-40ef-9760-b65145a33e72 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:13.355221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cbfc5-c302-40ef-9760-b65145a33e72 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:13.355221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78389d-c4f0-4f37-99a6-41fcd979703d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:13.355221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78389d-c4f0-4f37-99a6-41fcd979703d service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:13.355221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78389d-c4f0-4f37-99a6-41fcd979703d service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:13.355612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78389d-c4f0-4f37-99a6-41fcd979703d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:13.377518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78389d-c4f0-4f37-99a6-41fcd979703d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:13.379273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78389d-c4f0-4f37-99a6-41fcd979703d service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:13.390949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 817/1645] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:13.589598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a324e94f4e1b4199b684514b8818dda4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:13.593116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a324e94f4e1b4199b684514b8818dda4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:13.612117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a324e94f4e1b4199b684514b8818dda4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:13.645603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a324e94f4e1b4199b684514b8818dda4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:13.844510 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:13.906623 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a324e94f4e1b4199b684514b8818dda4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:13.918035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 829/1646] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:04:12 2022] DELETE /identity/v3/users/a324e94f4e1b4199b684514b8818dda4 => generated 0 bytes in 1017 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:13.927845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354270e-7754-4864-aa33-7e4250f6ec43 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:13.928806 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354270e-7754-4864-aa33-7e4250f6ec43 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:13.929450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354270e-7754-4864-aa33-7e4250f6ec43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:13.969541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-2354270e-7754-4864-aa33-7e4250f6ec43 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:13.972835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2354270e-7754-4864-aa33-7e4250f6ec43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:14.128320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 818/1647] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:14.144464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:14.146281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:14.267901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84812a-f2e0-47fe-8816-9038a1dfabe7 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:14.274285 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5175d-b592-4aec-bb47-031571411689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:14.275192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5175d-b592-4aec-bb47-031571411689 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:14.275785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5175d-b592-4aec-bb47-031571411689 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:14.276343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5175d-b592-4aec-bb47-031571411689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:14.293154 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5175d-b592-4aec-bb47-031571411689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:14.295298 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5175d-b592-4aec-bb47-031571411689 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:14.314992 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 830/1648] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:16.099117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354270e-7754-4864-aa33-7e4250f6ec43 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:16.102023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2354270e-7754-4864-aa33-7e4250f6ec43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:16.127503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354270e-7754-4864-aa33-7e4250f6ec43 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:16.139561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caec98-00de-4679-a700-a63d551ed2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:16.141002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caec98-00de-4679-a700-a63d551ed2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:16.141555 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caec98-00de-4679-a700-a63d551ed2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:16.142640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caec98-00de-4679-a700-a63d551ed2ab admin admin] PATH_INFO: `/v3/projects/0645a63ae797444585edd4eb2924f1e6` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:16.148448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caec98-00de-4679-a700-a63d551ed2ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0645a63ae797444585edd4eb2924f1e6)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:16.150800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caec98-00de-4679-a700-a63d551ed2ab admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:16.485838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-47caec98-00de-4679-a700-a63d551ed2ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0645a63ae797444585edd4eb2924f1e6', 'request_id': 'req-47caec98-00de-4679-a700-a63d551ed2ab'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:16.759039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-47caec98-00de-4679-a700-a63d551ed2ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0645a63ae797444585edd4eb2924f1e6', 'request_id': 'req-47caec98-00de-4679-a700-a63d551ed2ab'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:16.771359 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-47caec98-00de-4679-a700-a63d551ed2ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0645a63ae797444585edd4eb2924f1e6', 'request_id': 'req-47caec98-00de-4679-a700-a63d551ed2ab'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:16.837579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-47caec98-00de-4679-a700-a63d551ed2ab admin admin] The token cache is being invalidate because project 0645a63ae797444585edd4eb2924f1e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:04:16.838579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-47caec98-00de-4679-a700-a63d551ed2ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:16.839419 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-47caec98-00de-4679-a700-a63d551ed2ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:16.843172 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 819/1649] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:04:16 2022] DELETE /identity/v3/projects/0645a63ae797444585edd4eb2924f1e6 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:16.850484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87394201-7797-414c-8360-e436ab644163 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:16.851905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87394201-7797-414c-8360-e436ab644163 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:16.852917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87394201-7797-414c-8360-e436ab644163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:16.908150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-87394201-7797-414c-8360-e436ab644163 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:16.913595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-87394201-7797-414c-8360-e436ab644163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:17.235000 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 831/1650] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:17.242302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ef3db-914a-448c-9bca-45154f4843df None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:17.244337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ef3db-914a-448c-9bca-45154f4843df None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:17.245033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ef3db-914a-448c-9bca-45154f4843df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:17.386102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ca9ef3db-914a-448c-9bca-45154f4843df None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:17.396034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ca9ef3db-914a-448c-9bca-45154f4843df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:17.407760 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:17.560891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 820/1651] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:17.560978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87394201-7797-414c-8360-e436ab644163 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:17.582457 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87394201-7797-414c-8360-e436ab644163 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:17.589244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972a130-af85-47bf-aab9-2c305dbee28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:17.590297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972a130-af85-47bf-aab9-2c305dbee28c admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:17.590787 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972a130-af85-47bf-aab9-2c305dbee28c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:17.591219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972a130-af85-47bf-aab9-2c305dbee28c admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:17.594804 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972a130-af85-47bf-aab9-2c305dbee28c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:04:17.595733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972a130-af85-47bf-aab9-2c305dbee28c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:17.598751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972a130-af85-47bf-aab9-2c305dbee28c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:17.622073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 832/1652] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:04:17.629797 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ef3db-914a-448c-9bca-45154f4843df None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:17.675300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ef3db-914a-448c-9bca-45154f4843df None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:17.687143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d770b-054a-45a5-b6e0-dca8284bdab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:17.688136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d770b-054a-45a5-b6e0-dca8284bdab0 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:17.689495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d770b-054a-45a5-b6e0-dca8284bdab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:17.690045 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d770b-054a-45a5-b6e0-dca8284bdab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:17.691694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d770b-054a-45a5-b6e0-dca8284bdab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:17.696750 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d770b-054a-45a5-b6e0-dca8284bdab0 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:17.998461 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 821/1653] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:17 2022] POST /identity/v3/projects => generated 403 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:04:17.999853 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972a130-af85-47bf-aab9-2c305dbee28c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:18.017376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972a130-af85-47bf-aab9-2c305dbee28c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:18.208326 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70c29a-1782-495c-83a7-2f8ecd1bfb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:18.209058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70c29a-1782-495c-83a7-2f8ecd1bfb01 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:18.209346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70c29a-1782-495c-83a7-2f8ecd1bfb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:18.209633 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70c29a-1782-495c-83a7-2f8ecd1bfb01 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:18.210483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70c29a-1782-495c-83a7-2f8ecd1bfb01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:18.217674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70c29a-1782-495c-83a7-2f8ecd1bfb01 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:18.366929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 833/1654] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:17 2022] POST /identity/v3/users => generated 302 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:18.367177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d770b-054a-45a5-b6e0-dca8284bdab0 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:18.405942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d770b-054a-45a5-b6e0-dca8284bdab0 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:18.430895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ae579-acd8-4e4f-881f-87f1827bf361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:18.432409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ae579-acd8-4e4f-881f-87f1827bf361 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:18.432879 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ae579-acd8-4e4f-881f-87f1827bf361 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:18.433312 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ae579-acd8-4e4f-881f-87f1827bf361 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:18.434226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ae579-acd8-4e4f-881f-87f1827bf361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:18.439174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ae579-acd8-4e4f-881f-87f1827bf361 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:18.498309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 822/1655] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:04:18.500830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70c29a-1782-495c-83a7-2f8ecd1bfb01 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:18.522056 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70c29a-1782-495c-83a7-2f8ecd1bfb01 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:18.551979 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09601b0-23ad-4be1-bf7a-6e97dc5f6e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:18.555672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09601b0-23ad-4be1-bf7a-6e97dc5f6e30 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:18.556208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09601b0-23ad-4be1-bf7a-6e97dc5f6e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:18.556505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09601b0-23ad-4be1-bf7a-6e97dc5f6e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:18.558801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09601b0-23ad-4be1-bf7a-6e97dc5f6e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:18.568035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09601b0-23ad-4be1-bf7a-6e97dc5f6e30 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:18.623608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 834/1656] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:18.628961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ae579-acd8-4e4f-881f-87f1827bf361 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:18.651692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ae579-acd8-4e4f-881f-87f1827bf361 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:18.685736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdaf8b4-3ef1-4016-832e-c8a71aafec1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:18.687135 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdaf8b4-3ef1-4016-832e-c8a71aafec1e admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:18.687438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdaf8b4-3ef1-4016-832e-c8a71aafec1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:18.687731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdaf8b4-3ef1-4016-832e-c8a71aafec1e admin admin] PATH_INFO: `/v3/projects/901a8a9740cb4db8af03be75c133078c/users/c72918a324c64c83bfaf0ad34e60625a/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:18.695043 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:18.743448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdaf8b4-3ef1-4016-832e-c8a71aafec1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=901a8a9740cb4db8af03be75c133078c, user_id=c72918a324c64c83bfaf0ad34e60625a, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:18.746399 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdaf8b4-3ef1-4016-832e-c8a71aafec1e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:18.941004 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 823/1657] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:18 2022] PUT /identity/v3/projects/901a8a9740cb4db8af03be75c133078c/users/c72918a324c64c83bfaf0ad34e60625a/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:04:18.945573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816869-6879-4ebf-a655-5b188c2a486c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:18.946317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816869-6879-4ebf-a655-5b188c2a486c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:18.948151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816869-6879-4ebf-a655-5b188c2a486c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:19.019531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-c5816869-6879-4ebf-a655-5b188c2a486c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:19.024640 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c5816869-6879-4ebf-a655-5b188c2a486c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:19.233092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 835/1658] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:19.263428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdaf8b4-3ef1-4016-832e-c8a71aafec1e admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:19.266306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8fdaf8b4-3ef1-4016-832e-c8a71aafec1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:19.446595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdaf8b4-3ef1-4016-832e-c8a71aafec1e admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:19.453108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286f8e4-36be-4d2f-8c9d-4a8232bda109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:19.454116 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286f8e4-36be-4d2f-8c9d-4a8232bda109 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:19.455050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286f8e4-36be-4d2f-8c9d-4a8232bda109 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:19.455413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286f8e4-36be-4d2f-8c9d-4a8232bda109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:19.476280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0286f8e4-36be-4d2f-8c9d-4a8232bda109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:19.479071 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0286f8e4-36be-4d2f-8c9d-4a8232bda109 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:19.512566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 824/1659] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:20.722090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:20.924973 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:23.065721 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:23.970005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33de5dc-2320-42a8-88be-6124fd785856 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:23.970578 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33de5dc-2320-42a8-88be-6124fd785856 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:23.970991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33de5dc-2320-42a8-88be-6124fd785856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:24.049539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-b33de5dc-2320-42a8-88be-6124fd785856 None None] MFA Rules not processed for user `592de777b94148b791e18bf898386f34`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:24.051260 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b33de5dc-2320-42a8-88be-6124fd785856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:24.252371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:24.319955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 836/1660] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:24.331089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286f8e4-36be-4d2f-8c9d-4a8232bda109 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:24.333379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0286f8e4-36be-4d2f-8c9d-4a8232bda109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:24.491978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286f8e4-36be-4d2f-8c9d-4a8232bda109 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:24.511452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda4f11-2385-463c-b0ad-7f28c95f908d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:24.514704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda4f11-2385-463c-b0ad-7f28c95f908d service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:24.517304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda4f11-2385-463c-b0ad-7f28c95f908d service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:24.518430 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda4f11-2385-463c-b0ad-7f28c95f908d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:24.563874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda4f11-2385-463c-b0ad-7f28c95f908d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:24.569801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda4f11-2385-463c-b0ad-7f28c95f908d service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:24.607741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 825/1661] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:25.312681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961cb1f-dd33-41d3-8e2e-a8d500d8aea3 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:25.313986 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961cb1f-dd33-41d3-8e2e-a8d500d8aea3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:25.313986 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961cb1f-dd33-41d3-8e2e-a8d500d8aea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:25.401505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-2961cb1f-dd33-41d3-8e2e-a8d500d8aea3 None None] MFA Rules not processed for user `c72918a324c64c83bfaf0ad34e60625a`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:25.402827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-2961cb1f-dd33-41d3-8e2e-a8d500d8aea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:25.603761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 837/1662] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:25.605544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda4f11-2385-463c-b0ad-7f28c95f908d service nova] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:25.644032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda4f11-2385-463c-b0ad-7f28c95f908d service nova] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:25.660869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f4f49-5697-4fb0-848e-e8a72f6b1914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:25.661571 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f4f49-5697-4fb0-848e-e8a72f6b1914 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:25.661863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f4f49-5697-4fb0-848e-e8a72f6b1914 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:25.662140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f4f49-5697-4fb0-848e-e8a72f6b1914 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:25.663065 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f4f49-5697-4fb0-848e-e8a72f6b1914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:25.665571 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f4f49-5697-4fb0-848e-e8a72f6b1914 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:25.917178 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 826/1663] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:25 2022] POST /identity/v3/projects => generated 405 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:25.917824 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961cb1f-dd33-41d3-8e2e-a8d500d8aea3 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:25.939820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961cb1f-dd33-41d3-8e2e-a8d500d8aea3 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:26.217389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21a4e8-575b-4564-8ae3-0dfe20fe3d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:26.218120 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21a4e8-575b-4564-8ae3-0dfe20fe3d51 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:26.219200 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21a4e8-575b-4564-8ae3-0dfe20fe3d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:26.219558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21a4e8-575b-4564-8ae3-0dfe20fe3d51 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:26.220699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21a4e8-575b-4564-8ae3-0dfe20fe3d51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:26.223322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21a4e8-575b-4564-8ae3-0dfe20fe3d51 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:26.390237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 838/1664] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:25 2022] POST /identity/v3/users => generated 309 bytes in 473 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:04:26.397118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f4f49-5697-4fb0-848e-e8a72f6b1914 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:26.434018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f4f49-5697-4fb0-848e-e8a72f6b1914 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:26.462444 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8186a6-7dad-407a-b1f2-193aa2027e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:26.463699 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8186a6-7dad-407a-b1f2-193aa2027e5b admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:26.464204 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8186a6-7dad-407a-b1f2-193aa2027e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:26.464706 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8186a6-7dad-407a-b1f2-193aa2027e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:26.466161 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8186a6-7dad-407a-b1f2-193aa2027e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:26.478561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8186a6-7dad-407a-b1f2-193aa2027e5b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:26.515983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 827/1665] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:26.520450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21a4e8-575b-4564-8ae3-0dfe20fe3d51 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:26.536569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21a4e8-575b-4564-8ae3-0dfe20fe3d51 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:26.554762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a1112-4c41-4dc0-8c7b-e33d2ef01436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:26.556367 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a1112-4c41-4dc0-8c7b-e33d2ef01436 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:26.556835 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a1112-4c41-4dc0-8c7b-e33d2ef01436 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:26.557285 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a1112-4c41-4dc0-8c7b-e33d2ef01436 admin admin] PATH_INFO: `/v3/projects/3726327cbac64e5a86392b6d2ae2cf4b/users/715be1c4d7fd4b14ae02e0183a2c4d6e/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:26.606301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a1112-4c41-4dc0-8c7b-e33d2ef01436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3726327cbac64e5a86392b6d2ae2cf4b, user_id=715be1c4d7fd4b14ae02e0183a2c4d6e, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:26.610967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a1112-4c41-4dc0-8c7b-e33d2ef01436 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:26.720894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 839/1666] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:26 2022] PUT /identity/v3/projects/3726327cbac64e5a86392b6d2ae2cf4b/users/715be1c4d7fd4b14ae02e0183a2c4d6e/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:26.721729 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8186a6-7dad-407a-b1f2-193aa2027e5b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:26.743795 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8186a6-7dad-407a-b1f2-193aa2027e5b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:26.816657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a7d68-9ccc-45aa-b8c8-eb39e972973e None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:26.817404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a7d68-9ccc-45aa-b8c8-eb39e972973e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:26.817879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a7d68-9ccc-45aa-b8c8-eb39e972973e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:26.876918 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-a91a7d68-9ccc-45aa-b8c8-eb39e972973e None None] MFA Rules not processed for user `7e3f168283ba4f159948830f396ae7cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:26.878998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a91a7d68-9ccc-45aa-b8c8-eb39e972973e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:26.968894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a6080-7363-44e0-ab77-3693f1cde093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:26.969076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a6080-7363-44e0-ab77-3693f1cde093 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:26.969290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a6080-7363-44e0-ab77-3693f1cde093 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:26.970007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a6080-7363-44e0-ab77-3693f1cde093 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:26.970995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a6080-7363-44e0-ab77-3693f1cde093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:26.975947 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a6080-7363-44e0-ab77-3693f1cde093 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:27.008050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 828/1667] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:04:27.011544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a6080-7363-44e0-ab77-3693f1cde093 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:27.020987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:27.051196 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a6080-7363-44e0-ab77-3693f1cde093 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:27.071538 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133bdd2-c70a-4efa-aa0a-4b7d2f6b2643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:27.073533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133bdd2-c70a-4efa-aa0a-4b7d2f6b2643 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:27.074085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133bdd2-c70a-4efa-aa0a-4b7d2f6b2643 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:27.074564 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133bdd2-c70a-4efa-aa0a-4b7d2f6b2643 admin admin] PATH_INFO: `/v3/projects/3726327cbac64e5a86392b6d2ae2cf4b/users/715be1c4d7fd4b14ae02e0183a2c4d6e/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:27.082171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133bdd2-c70a-4efa-aa0a-4b7d2f6b2643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3726327cbac64e5a86392b6d2ae2cf4b, user_id=715be1c4d7fd4b14ae02e0183a2c4d6e, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:27.120426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 840/1668] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:27.120539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133bdd2-c70a-4efa-aa0a-4b7d2f6b2643 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:27.124523 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fa7f9-1033-429e-819e-7860e4b6158c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:27.124941 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fa7f9-1033-429e-819e-7860e4b6158c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:27.125378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fa7f9-1033-429e-819e-7860e4b6158c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:27.175021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 829/1669] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:27 2022] PUT /identity/v3/projects/3726327cbac64e5a86392b6d2ae2cf4b/users/715be1c4d7fd4b14ae02e0183a2c4d6e/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:27.200240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-462fa7f9-1033-429e-819e-7860e4b6158c None None] MFA Rules not processed for user `7e3f168283ba4f159948830f396ae7cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:27.201883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-462fa7f9-1033-429e-819e-7860e4b6158c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:27.573301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 841/1670] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:04:27.598153 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133bdd2-c70a-4efa-aa0a-4b7d2f6b2643 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:27.599806 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f133bdd2-c70a-4efa-aa0a-4b7d2f6b2643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:27.610927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:27.790823 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133bdd2-c70a-4efa-aa0a-4b7d2f6b2643 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:27.813968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183ebca-2226-46bd-8c56-c2a8729b7e54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:27.815110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183ebca-2226-46bd-8c56-c2a8729b7e54 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:27.815611 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183ebca-2226-46bd-8c56-c2a8729b7e54 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:27.816309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183ebca-2226-46bd-8c56-c2a8729b7e54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:27.848306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183ebca-2226-46bd-8c56-c2a8729b7e54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:27.850231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183ebca-2226-46bd-8c56-c2a8729b7e54 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:27.893622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 830/1671] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:29.047881 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59ce13-a765-41e2-85cf-1783f21e2c9f None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:29.049182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59ce13-a765-41e2-85cf-1783f21e2c9f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:29.049795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59ce13-a765-41e2-85cf-1783f21e2c9f None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:29.052876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 842/1672] 10.222.0.17 () {54 vars in 912 bytes} [Mon Jul 25 19:04:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:04:29.060314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cbc6c-ee28-49e2-850d-8c1c7d9036fe None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:29.060838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cbc6c-ee28-49e2-850d-8c1c7d9036fe None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:29.060942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cbc6c-ee28-49e2-850d-8c1c7d9036fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:29.281352 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e94cbc6c-ee28-49e2-850d-8c1c7d9036fe None None] MFA Rules not processed for user `ffcf280bc6d54b2fb2d22634468ac6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:29.281352 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e94cbc6c-ee28-49e2-850d-8c1c7d9036fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:29.398241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 831/1673] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:04:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:29.659295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59ce13-a765-41e2-85cf-1783f21e2c9f None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:29.661166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-da59ce13-a765-41e2-85cf-1783f21e2c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:29.694923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59ce13-a765-41e2-85cf-1783f21e2c9f None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:29.710845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ffb1e-243f-4e5a-a098-986c0f8c91ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:29.711554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ffb1e-243f-4e5a-a098-986c0f8c91ca service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:29.711846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ffb1e-243f-4e5a-a098-986c0f8c91ca service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:29.712126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ffb1e-243f-4e5a-a098-986c0f8c91ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:29.750584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ffb1e-243f-4e5a-a098-986c0f8c91ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:29.750939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b106a-ea0b-4163-ac6d-f71416a625b8 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:29.751844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b106a-ea0b-4163-ac6d-f71416a625b8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:29.753262 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b106a-ea0b-4163-ac6d-f71416a625b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:29.783032 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ffb1e-243f-4e5a-a098-986c0f8c91ca service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:29.826380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 843/1674] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:04:29.872956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-506b106a-ea0b-4163-ac6d-f71416a625b8 None None] MFA Rules not processed for user `c99e02a845184f0aa206ba9f209ae4ef`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:29.882164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-506b106a-ea0b-4163-ac6d-f71416a625b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:30.103659 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 832/1675] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:30.126078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ffb1e-243f-4e5a-a098-986c0f8c91ca service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:30.153979 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ffb1e-243f-4e5a-a098-986c0f8c91ca service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:30.167645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ace59e-3c99-4419-bcf0-772877e874bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:30.173760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ace59e-3c99-4419-bcf0-772877e874bc service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:30.174446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ace59e-3c99-4419-bcf0-772877e874bc service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:30.174926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ace59e-3c99-4419-bcf0-772877e874bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:30.196801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ace59e-3c99-4419-bcf0-772877e874bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:30.198788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ace59e-3c99-4419-bcf0-772877e874bc service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:30.231618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 844/1676] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:31.197383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bad5a-7ceb-45a4-985c-55f19fd81413 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:31.197862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bad5a-7ceb-45a4-985c-55f19fd81413 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:31.198006 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bad5a-7ceb-45a4-985c-55f19fd81413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:31.280528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-305bad5a-7ceb-45a4-985c-55f19fd81413 None None] MFA Rules not processed for user `c0464996da4f44008ca8b91c837ac22d`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:31.282476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-305bad5a-7ceb-45a4-985c-55f19fd81413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:31.294995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:31.389957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb2737-f164-4a46-9e0f-0018c5ffc18f None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:31.390580 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb2737-f164-4a46-9e0f-0018c5ffc18f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:31.390672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb2737-f164-4a46-9e0f-0018c5ffc18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:31.456727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-a6bb2737-f164-4a46-9e0f-0018c5ffc18f None None] MFA Rules not processed for user `715be1c4d7fd4b14ae02e0183a2c4d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:31.460264 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a6bb2737-f164-4a46-9e0f-0018c5ffc18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:31.562572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 833/1677] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:31.569382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740731ce-097f-40eb-b0b1-c7c56366c67c None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:31.572816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740731ce-097f-40eb-b0b1-c7c56366c67c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:31.574472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740731ce-097f-40eb-b0b1-c7c56366c67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:31.619134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-740731ce-097f-40eb-b0b1-c7c56366c67c None None] MFA Rules not processed for user `c0464996da4f44008ca8b91c837ac22d`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:31.620438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-740731ce-097f-40eb-b0b1-c7c56366c67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:31.642590 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 834/1678] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:31.663785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740731ce-097f-40eb-b0b1-c7c56366c67c None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:31.672262 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 845/1679] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:31 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:31.730022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740731ce-097f-40eb-b0b1-c7c56366c67c None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:31.771415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70ce47-27c3-42e3-a302-41948c030fc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:31.771415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70ce47-27c3-42e3-a302-41948c030fc6 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:31.772404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70ce47-27c3-42e3-a302-41948c030fc6 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:31.772404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70ce47-27c3-42e3-a302-41948c030fc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:31.815796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70ce47-27c3-42e3-a302-41948c030fc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:31.817549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70ce47-27c3-42e3-a302-41948c030fc6 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:31.855309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 835/1680] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:32.470695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77a455-6a78-4321-81f3-8886efd93ba0 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:32.473641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77a455-6a78-4321-81f3-8886efd93ba0 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:32.473641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77a455-6a78-4321-81f3-8886efd93ba0 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:32.474692 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 846/1681] 10.222.0.17 () {54 vars in 961 bytes} [Mon Jul 25 19:04:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:04:32.517485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ef7ae-4b66-4173-b023-b8d75546eff6 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:32.520987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1d200-568e-44a5-b060-16b2ec1c4671 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:32.520987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1d200-568e-44a5-b060-16b2ec1c4671 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:32.520987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1d200-568e-44a5-b060-16b2ec1c4671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:32.523292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ef7ae-4b66-4173-b023-b8d75546eff6 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:32.524096 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ef7ae-4b66-4173-b023-b8d75546eff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:32.607192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-38c1d200-568e-44a5-b060-16b2ec1c4671 None None] MFA Rules not processed for user `7e3f168283ba4f159948830f396ae7cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:32.616514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-38c1d200-568e-44a5-b060-16b2ec1c4671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:32.630732 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ab7ef7ae-4b66-4173-b023-b8d75546eff6 None None] MFA Rules not processed for user `f5027a60da214afda9144b5ae6cce15b`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:32.630732 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ab7ef7ae-4b66-4173-b023-b8d75546eff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:32.654092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 836/1682] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:32.907975 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:33.111334 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 847/1683] 10.222.0.17 () {58 vars in 1058 bytes} [Mon Jul 25 19:04:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 595 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:33.137405 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1d200-568e-44a5-b060-16b2ec1c4671 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:33.172142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1d200-568e-44a5-b060-16b2ec1c4671 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:33.222109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1bc6b-1d71-4d06-bf09-00ceaa08f2e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:33.222933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1bc6b-1d71-4d06-bf09-00ceaa08f2e8 service placement] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:33.223316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1bc6b-1d71-4d06-bf09-00ceaa08f2e8 service placement] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:33.223766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1bc6b-1d71-4d06-bf09-00ceaa08f2e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:33.238467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e2a1bc6b-1d71-4d06-bf09-00ceaa08f2e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:33.299534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1bc6b-1d71-4d06-bf09-00ceaa08f2e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:33.310000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1bc6b-1d71-4d06-bf09-00ceaa08f2e8 service placement] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:33.352873 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 837/1684] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:33.697088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10be99-ba20-42ae-98b9-3aa9a7448c6c None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:33.701929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10be99-ba20-42ae-98b9-3aa9a7448c6c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:33.701929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10be99-ba20-42ae-98b9-3aa9a7448c6c None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:33.702068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 848/1685] 10.222.0.17 () {54 vars in 912 bytes} [Mon Jul 25 19:04:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:04:33.711972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17b30f-5aee-47f9-baf4-aef40056d14f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:33.712646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17b30f-5aee-47f9-baf4-aef40056d14f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:33.712882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17b30f-5aee-47f9-baf4-aef40056d14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:33.767966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-fe17b30f-5aee-47f9-baf4-aef40056d14f None None] MFA Rules not processed for user `ffcf280bc6d54b2fb2d22634468ac6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:33.771950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-fe17b30f-5aee-47f9-baf4-aef40056d14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:33.820295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 838/1686] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:04:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:04:33.833803 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10be99-ba20-42ae-98b9-3aa9a7448c6c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:33.980228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10be99-ba20-42ae-98b9-3aa9a7448c6c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:33.980228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56f17a-9430-4099-ba40-886a13ab421d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:33.980228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56f17a-9430-4099-ba40-886a13ab421d service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:33.980228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56f17a-9430-4099-ba40-886a13ab421d service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:33.982312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56f17a-9430-4099-ba40-886a13ab421d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:33.982312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56f17a-9430-4099-ba40-886a13ab421d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:33.982312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56f17a-9430-4099-ba40-886a13ab421d service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:34.055793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 849/1687] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:35.855869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eedf29-837a-4584-9f0d-7fbd55423f82 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:35.856669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eedf29-837a-4584-9f0d-7fbd55423f82 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:35.856669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eedf29-837a-4584-9f0d-7fbd55423f82 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:35.858790 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 839/1688] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 19:04:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:04:35.924816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b82d60-a26f-42e9-bd7f-123016c12e9c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:35.924816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b82d60-a26f-42e9-bd7f-123016c12e9c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:35.924816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b82d60-a26f-42e9-bd7f-123016c12e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:35.996262 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-e3b82d60-a26f-42e9-bd7f-123016c12e9c None None] MFA Rules not processed for user `f5027a60da214afda9144b5ae6cce15b`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:36.004409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e3b82d60-a26f-42e9-bd7f-123016c12e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:36.063949 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 850/1689] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 19:04:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:04:36.226854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eedf29-837a-4584-9f0d-7fbd55423f82 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:36.229606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-50eedf29-837a-4584-9f0d-7fbd55423f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:36.265019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eedf29-837a-4584-9f0d-7fbd55423f82 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:36.282649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c1130-32f2-4031-9e25-6158d09c408d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:36.283628 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c1130-32f2-4031-9e25-6158d09c408d service placement] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:36.284085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c1130-32f2-4031-9e25-6158d09c408d service placement] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:36.284539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c1130-32f2-4031-9e25-6158d09c408d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:36.308700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c1130-32f2-4031-9e25-6158d09c408d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:36.314587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c1130-32f2-4031-9e25-6158d09c408d service placement] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:36.355123 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 840/1690] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:37.780555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46ce9f-bd48-4da8-8467-4f357353fca8 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:37.781285 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46ce9f-bd48-4da8-8467-4f357353fca8 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:37.782870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46ce9f-bd48-4da8-8467-4f357353fca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:37.855619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-da46ce9f-bd48-4da8-8467-4f357353fca8 None None] MFA Rules not processed for user `c99e02a845184f0aa206ba9f209ae4ef`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:37.855619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-da46ce9f-bd48-4da8-8467-4f357353fca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:37.896915 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 851/1691] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:38.190831 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec9da2-8f4c-4592-8bdc-64489cdb52d3 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:38.191604 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec9da2-8f4c-4592-8bdc-64489cdb52d3 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:38.191604 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec9da2-8f4c-4592-8bdc-64489cdb52d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:38.252678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3eec9da2-8f4c-4592-8bdc-64489cdb52d3 None None] MFA Rules not processed for user `592de777b94148b791e18bf898386f34`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:38.259457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3eec9da2-8f4c-4592-8bdc-64489cdb52d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:38.454866 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 841/1692] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:39.643104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46ce9f-bd48-4da8-8467-4f357353fca8 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:39.649564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-da46ce9f-bd48-4da8-8467-4f357353fca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:39.781090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec9da2-8f4c-4592-8bdc-64489cdb52d3 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:39.784124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3eec9da2-8f4c-4592-8bdc-64489cdb52d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:40.002728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46ce9f-bd48-4da8-8467-4f357353fca8 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:40.036461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:40.037513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:40.037981 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:40.038378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] PATH_INFO: `/v3/users/9481d8c7a5a743aa8ad0e66d9a718cc4` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:40.046256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9481d8c7a5a743aa8ad0e66d9a718cc4)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:40.050406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:40.060889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:40.161597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec9da2-8f4c-4592-8bdc-64489cdb52d3 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:40.182692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d5cb9-dbe3-4618-96f6-d3b817775b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:40.184066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5cb9-dbe3-4618-96f6-d3b817775b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:40.187411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5cb9-dbe3-4618-96f6-d3b817775b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:40.190965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5cb9-dbe3-4618-96f6-d3b817775b90 admin admin] PATH_INFO: `/v3/users/88f57dc362e746f98cea2b84e8b1e351` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:40.190965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d5cb9-dbe3-4618-96f6-d3b817775b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88f57dc362e746f98cea2b84e8b1e351)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:40.194273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: WARNING py.warnings [None req-ee2d5cb9-dbe3-4618-96f6-d3b817775b90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:04:40.194273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: warnings.warn(msg) Jul 25 19:04:40.194273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: Jul 25 19:04:40.198291 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d5cb9-dbe3-4618-96f6-d3b817775b90 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:40.448869 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9481d8c7a5a743aa8ad0e66d9a718cc4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:40.453576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9481d8c7a5a743aa8ad0e66d9a718cc4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:40.466879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9481d8c7a5a743aa8ad0e66d9a718cc4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:40.525672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9481d8c7a5a743aa8ad0e66d9a718cc4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:40.585279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ee2d5cb9-dbe3-4618-96f6-d3b817775b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88f57dc362e746f98cea2b84e8b1e351'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:40.590453 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ee2d5cb9-dbe3-4618-96f6-d3b817775b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88f57dc362e746f98cea2b84e8b1e351'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:40.595381 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ee2d5cb9-dbe3-4618-96f6-d3b817775b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88f57dc362e746f98cea2b84e8b1e351'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:40.812533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9481d8c7a5a743aa8ad0e66d9a718cc4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:40.830270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 852/1693] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:04:39 2022] DELETE /identity/v3/users/9481d8c7a5a743aa8ad0e66d9a718cc4 => generated 0 bytes in 1188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:40.831740 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:40.836171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:40.899283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:41.005295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ee2d5cb9-dbe3-4618-96f6-d3b817775b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88f57dc362e746f98cea2b84e8b1e351'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:41.073202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7b210-91ce-4d4f-9cfb-9be1fd22429c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:41.096092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:41.101131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:41.113433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:41.113898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ee2d5cb9-dbe3-4618-96f6-d3b817775b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88f57dc362e746f98cea2b84e8b1e351'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:41.114049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] PATH_INFO: `/v3/users/c72918a324c64c83bfaf0ad34e60625a` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:41.114049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c72918a324c64c83bfaf0ad34e60625a)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:41.119253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:41.146591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 842/1694] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:04:39 2022] DELETE /identity/v3/users/88f57dc362e746f98cea2b84e8b1e351 => generated 0 bytes in 1366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:04:41.153639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f7d0e-2d41-4e1c-8761-d2250f79439c None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:41.153639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f7d0e-2d41-4e1c-8761-d2250f79439c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:41.153639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f7d0e-2d41-4e1c-8761-d2250f79439c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:41.349609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-854f7d0e-2d41-4e1c-8761-d2250f79439c None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:41.351023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-854f7d0e-2d41-4e1c-8761-d2250f79439c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:41.545521 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c72918a324c64c83bfaf0ad34e60625a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:41.546436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c72918a324c64c83bfaf0ad34e60625a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:41.587857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c72918a324c64c83bfaf0ad34e60625a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:41.659069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c72918a324c64c83bfaf0ad34e60625a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:41.698268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 843/1695] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:04:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 551 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:41.704769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1ca85-ec0c-4e80-b6b0-4fdd17900cb5 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:41.705309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1ca85-ec0c-4e80-b6b0-4fdd17900cb5 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:41.705652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1ca85-ec0c-4e80-b6b0-4fdd17900cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:41.789632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-5ba1ca85-ec0c-4e80-b6b0-4fdd17900cb5 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:41.792346 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5ba1ca85-ec0c-4e80-b6b0-4fdd17900cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:41.887577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:41.933237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 844/1696] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:41.936114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776746c-3b00-4679-8fc1-02a244481349 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:41.947592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776746c-3b00-4679-8fc1-02a244481349 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:41.948069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776746c-3b00-4679-8fc1-02a244481349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:41.970274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c72918a324c64c83bfaf0ad34e60625a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:41.990779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 853/1697] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:04:40 2022] DELETE /identity/v3/users/c72918a324c64c83bfaf0ad34e60625a => generated 0 bytes in 1160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:41.992419 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:42.002686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:42.098239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-1776746c-3b00-4679-8fc1-02a244481349 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:42.099563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1776746c-3b00-4679-8fc1-02a244481349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:42.333572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832facac-1518-4f75-bd30-d0735a7539c9 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:42.347530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2bd84-d78c-4533-a0fb-a9c1409c208d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:42.352009 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2bd84-d78c-4533-a0fb-a9c1409c208d service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:42.352335 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2bd84-d78c-4533-a0fb-a9c1409c208d service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:42.353385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2bd84-d78c-4533-a0fb-a9c1409c208d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:42.361103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-19c2bd84-d78c-4533-a0fb-a9c1409c208d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:42.505640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 845/1698] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 571 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:42.519181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776746c-3b00-4679-8fc1-02a244481349 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:42.539750 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1776746c-3b00-4679-8fc1-02a244481349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:42.560373 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2bd84-d78c-4533-a0fb-a9c1409c208d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:42.565542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2bd84-d78c-4533-a0fb-a9c1409c208d service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:42.600966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 854/1699] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:42.686517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2bd84-d78c-4533-a0fb-a9c1409c208d service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:42.721762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2bd84-d78c-4533-a0fb-a9c1409c208d service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:42.755148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec7c8f-85dc-4eb9-8bfd-05b900d1fc53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:42.756204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec7c8f-85dc-4eb9-8bfd-05b900d1fc53 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:42.756582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec7c8f-85dc-4eb9-8bfd-05b900d1fc53 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:42.759321 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec7c8f-85dc-4eb9-8bfd-05b900d1fc53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:42.835880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec7c8f-85dc-4eb9-8bfd-05b900d1fc53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:42.849866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec7c8f-85dc-4eb9-8bfd-05b900d1fc53 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:42.931584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 855/1700] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:04:43.005192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776746c-3b00-4679-8fc1-02a244481349 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:43.028308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d175e-bef8-4b7d-ad00-97bedd89b203 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:43.030714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d175e-bef8-4b7d-ad00-97bedd89b203 service swift] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:43.030714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d175e-bef8-4b7d-ad00-97bedd89b203 service swift] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:43.032378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d175e-bef8-4b7d-ad00-97bedd89b203 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:43.039706 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4e2d175e-bef8-4b7d-ad00-97bedd89b203 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:43.280674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:43.311172 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d175e-bef8-4b7d-ad00-97bedd89b203 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:43.317200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d175e-bef8-4b7d-ad00-97bedd89b203 service swift] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:43.379343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 846/1701] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:04:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 861 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:04:43.993444 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c7187-5349-4b09-bbda-a949f46f7881 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:43.993893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c7187-5349-4b09-bbda-a949f46f7881 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:43.994356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c7187-5349-4b09-bbda-a949f46f7881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:44.092497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-255c7187-5349-4b09-bbda-a949f46f7881 None None] MFA Rules not processed for user `ffcf280bc6d54b2fb2d22634468ac6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:44.099833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-255c7187-5349-4b09-bbda-a949f46f7881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:44.196353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 856/1702] 10.222.0.88 () {58 vars in 1015 bytes} [Mon Jul 25 19:04:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:44.210236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d175e-bef8-4b7d-ad00-97bedd89b203 service swift] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:44.241568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d175e-bef8-4b7d-ad00-97bedd89b203 service swift] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:44.264009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e2092-4c74-4394-bd14-9fe91733fa75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:44.270083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e2092-4c74-4394-bd14-9fe91733fa75 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:44.276143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e2092-4c74-4394-bd14-9fe91733fa75 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:44.279201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e2092-4c74-4394-bd14-9fe91733fa75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:44.316046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e2092-4c74-4394-bd14-9fe91733fa75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:44.318223 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e2092-4c74-4394-bd14-9fe91733fa75 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:44.371482 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 847/1703] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:04:44.371482 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:45.646533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c7187-5349-4b09-bbda-a949f46f7881 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:45.651166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-255c7187-5349-4b09-bbda-a949f46f7881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:45.719575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c7187-5349-4b09-bbda-a949f46f7881 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:45.736558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:45.737026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:45.737566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:45.738100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] PATH_INFO: `/v3/projects/e02ab65920c04a8296db23c2ba37bd14` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:45.740706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e02ab65920c04a8296db23c2ba37bd14)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:45.742462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: WARNING py.warnings [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:04:45.742462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: warnings.warn(msg) Jul 25 19:04:45.742462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: Jul 25 19:04:45.744668 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:46.050746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e2092-4c74-4394-bd14-9fe91733fa75 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:46.055908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1e3e2092-4c74-4394-bd14-9fe91733fa75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:46.131237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e2092-4c74-4394-bd14-9fe91733fa75 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:46.191330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e02ab65920c04a8296db23c2ba37bd14', 'request_id': 'req-23df28e8-4f99-458e-9233-7058b622c9f5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:46.234368 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e02ab65920c04a8296db23c2ba37bd14', 'request_id': 'req-23df28e8-4f99-458e-9233-7058b622c9f5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:46.265282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e02ab65920c04a8296db23c2ba37bd14', 'request_id': 'req-23df28e8-4f99-458e-9233-7058b622c9f5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:46.396779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23f3f4-e830-412f-bcf6-072a62b982ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:46.397710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23f3f4-e830-412f-bcf6-072a62b982ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:46.398057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23f3f4-e830-412f-bcf6-072a62b982ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:46.398366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23f3f4-e830-412f-bcf6-072a62b982ef admin admin] PATH_INFO: `/v3/projects/31a9679793634657a266513b88ea7d88` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:46.404072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23f3f4-e830-412f-bcf6-072a62b982ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=31a9679793634657a266513b88ea7d88)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:46.404072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23f3f4-e830-412f-bcf6-072a62b982ef admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:46.419304 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] The token cache is being invalidate because project e02ab65920c04a8296db23c2ba37bd14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:04:46.421599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:46.427080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:46.427080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 857/1704] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:04:45 2022] DELETE /identity/v3/projects/e02ab65920c04a8296db23c2ba37bd14 => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:46.432660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:46.436723 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:46.473689 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df28e8-4f99-458e-9233-7058b622c9f5 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:46.487585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:46.495195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:46.495195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:46.495195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] PATH_INFO: `/v3/projects/901a8a9740cb4db8af03be75c133078c` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:46.495195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=901a8a9740cb4db8af03be75c133078c)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:46.496631 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:46.644394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4f23f3f4-e830-412f-bcf6-072a62b982ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31a9679793634657a266513b88ea7d88', 'request_id': 'req-4f23f3f4-e830-412f-bcf6-072a62b982ef'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:46.742142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901a8a9740cb4db8af03be75c133078c', 'request_id': 'req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:46.748280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4f23f3f4-e830-412f-bcf6-072a62b982ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31a9679793634657a266513b88ea7d88', 'request_id': 'req-4f23f3f4-e830-412f-bcf6-072a62b982ef'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:46.757908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4f23f3f4-e830-412f-bcf6-072a62b982ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31a9679793634657a266513b88ea7d88', 'request_id': 'req-4f23f3f4-e830-412f-bcf6-072a62b982ef'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:46.810359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901a8a9740cb4db8af03be75c133078c', 'request_id': 'req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:46.821283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4f23f3f4-e830-412f-bcf6-072a62b982ef admin admin] The token cache is being invalidate because project 31a9679793634657a266513b88ea7d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:04:46.823450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901a8a9740cb4db8af03be75c133078c', 'request_id': 'req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:46.824072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4f23f3f4-e830-412f-bcf6-072a62b982ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:46.827063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4f23f3f4-e830-412f-bcf6-072a62b982ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:46.840015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 848/1705] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:04:46 2022] DELETE /identity/v3/projects/31a9679793634657a266513b88ea7d88 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:46.850023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1e141-6461-4b9f-993c-68ae70798953 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:46.852451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1e141-6461-4b9f-993c-68ae70798953 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:46.854636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1e141-6461-4b9f-993c-68ae70798953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:46.938410 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-46a1e141-6461-4b9f-993c-68ae70798953 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:46.941763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-46a1e141-6461-4b9f-993c-68ae70798953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:47.115282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] The token cache is being invalidate because project 901a8a9740cb4db8af03be75c133078c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:04:47.118637 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:47.122563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:47.147287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 858/1706] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:04:46 2022] DELETE /identity/v3/projects/901a8a9740cb4db8af03be75c133078c => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:47.147287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:47.150035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:47.212250 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 849/1707] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:47.213325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1e141-6461-4b9f-993c-68ae70798953 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:47.218358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-46a1e141-6461-4b9f-993c-68ae70798953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:47.238480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5aac9-6fd5-4950-a2c9-2a6cbf61a489 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:47.252883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1e141-6461-4b9f-993c-68ae70798953 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:47.269936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:47.273000 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfedaf-f27d-4f15-a2f2-90e585f92a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:47.273250 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:47.282519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfedaf-f27d-4f15-a2f2-90e585f92a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:47.283504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfedaf-f27d-4f15-a2f2-90e585f92a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:47.284257 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfedaf-f27d-4f15-a2f2-90e585f92a67 admin admin] PATH_INFO: `/v3/users/9ee4896bed48485fa71517a20853311e` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:47.286150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfedaf-f27d-4f15-a2f2-90e585f92a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee4896bed48485fa71517a20853311e)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:47.286652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:47.287440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] PATH_INFO: `/v3/users/715be1c4d7fd4b14ae02e0183a2c4d6e` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:47.289297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfedaf-f27d-4f15-a2f2-90e585f92a67 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:47.294845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=715be1c4d7fd4b14ae02e0183a2c4d6e)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:47.340291 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:47.415744 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:47.594403 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:47.638129 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-babfedaf-f27d-4f15-a2f2-90e585f92a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee4896bed48485fa71517a20853311e'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:47.649164 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-babfedaf-f27d-4f15-a2f2-90e585f92a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee4896bed48485fa71517a20853311e'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:47.685199 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-babfedaf-f27d-4f15-a2f2-90e585f92a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee4896bed48485fa71517a20853311e'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:47.701597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '715be1c4d7fd4b14ae02e0183a2c4d6e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:47.702304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '715be1c4d7fd4b14ae02e0183a2c4d6e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:47.702956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '715be1c4d7fd4b14ae02e0183a2c4d6e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:47.757421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-babfedaf-f27d-4f15-a2f2-90e585f92a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee4896bed48485fa71517a20853311e'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:47.802205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '715be1c4d7fd4b14ae02e0183a2c4d6e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:47.856354 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '715be1c4d7fd4b14ae02e0183a2c4d6e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:47.904776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-babfedaf-f27d-4f15-a2f2-90e585f92a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee4896bed48485fa71517a20853311e'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:47.930138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 859/1708] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:04:47 2022] DELETE /identity/v3/users/9ee4896bed48485fa71517a20853311e => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:47.938965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b856ac-1cbe-4a55-9daf-0873f98ca9a7 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:47.942538 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b856ac-1cbe-4a55-9daf-0873f98ca9a7 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:47.942538 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b856ac-1cbe-4a55-9daf-0873f98ca9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:47.950997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 850/1709] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:04:47 2022] DELETE /identity/v3/users/715be1c4d7fd4b14ae02e0183a2c4d6e => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:48.036160 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-f3b856ac-1cbe-4a55-9daf-0873f98ca9a7 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:48.039369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f3b856ac-1cbe-4a55-9daf-0873f98ca9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:48.251468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 860/1710] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:48.258765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:48.265280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:48.314476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fc403-efdf-4952-a032-7025af8575d0 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:48.336610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b628543-439e-4e41-8647-0105bc6b5224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:48.351201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b628543-439e-4e41-8647-0105bc6b5224 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:48.363270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b628543-439e-4e41-8647-0105bc6b5224 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:48.372885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b628543-439e-4e41-8647-0105bc6b5224 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:48.372885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b628543-439e-4e41-8647-0105bc6b5224 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:04:48.372885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b628543-439e-4e41-8647-0105bc6b5224 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:48.375696 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b628543-439e-4e41-8647-0105bc6b5224 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:48.436913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 851/1711] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:04:48.439587 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b856ac-1cbe-4a55-9daf-0873f98ca9a7 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:48.460870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b856ac-1cbe-4a55-9daf-0873f98ca9a7 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:48.479456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e897a97-c66a-4b48-90f7-79c2383d084f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:48.482280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e897a97-c66a-4b48-90f7-79c2383d084f admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:48.483086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e897a97-c66a-4b48-90f7-79c2383d084f admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:48.483758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e897a97-c66a-4b48-90f7-79c2383d084f admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:48.487488 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e897a97-c66a-4b48-90f7-79c2383d084f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:48.492023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e897a97-c66a-4b48-90f7-79c2383d084f admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:48.663759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 861/1712] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:48 2022] POST /identity/v3/projects => generated 393 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:04:48.670832 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b628543-439e-4e41-8647-0105bc6b5224 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:48.697290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b628543-439e-4e41-8647-0105bc6b5224 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:48.961229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ba0bb-6a14-4850-b42c-e3fe9c5eac89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:48.961229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ba0bb-6a14-4850-b42c-e3fe9c5eac89 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:48.961229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ba0bb-6a14-4850-b42c-e3fe9c5eac89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:48.961229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ba0bb-6a14-4850-b42c-e3fe9c5eac89 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:48.961730 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ba0bb-6a14-4850-b42c-e3fe9c5eac89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:48.961730 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ba0bb-6a14-4850-b42c-e3fe9c5eac89 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:49.149706 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 852/1713] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:48 2022] POST /identity/v3/users => generated 297 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:49.151841 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e897a97-c66a-4b48-90f7-79c2383d084f admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:49.198366 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e897a97-c66a-4b48-90f7-79c2383d084f admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:49.217494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acbd99-2f23-4f43-913e-ad40cfdc19f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:49.217494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acbd99-2f23-4f43-913e-ad40cfdc19f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:49.217494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acbd99-2f23-4f43-913e-ad40cfdc19f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:49.217494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acbd99-2f23-4f43-913e-ad40cfdc19f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:49.219028 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1acbd99-2f23-4f43-913e-ad40cfdc19f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:49.223127 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1acbd99-2f23-4f43-913e-ad40cfdc19f2 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:49.268209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 862/1714] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:49.273185 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ba0bb-6a14-4850-b42c-e3fe9c5eac89 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:49.295084 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ba0bb-6a14-4850-b42c-e3fe9c5eac89 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:49.302112 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1d4e6-0a20-44b4-bddc-01a726d90e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:49.304284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1d4e6-0a20-44b4-bddc-01a726d90e1b admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:49.307983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1d4e6-0a20-44b4-bddc-01a726d90e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:49.307983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1d4e6-0a20-44b4-bddc-01a726d90e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:49.307983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1d4e6-0a20-44b4-bddc-01a726d90e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:49.309184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1d4e6-0a20-44b4-bddc-01a726d90e1b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:49.328760 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 853/1715] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:49.341677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acbd99-2f23-4f43-913e-ad40cfdc19f2 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:49.364748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acbd99-2f23-4f43-913e-ad40cfdc19f2 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:49.381854 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67427a38-a59f-4cb2-b022-a8cfed55167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:49.384496 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67427a38-a59f-4cb2-b022-a8cfed55167c admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:49.385783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67427a38-a59f-4cb2-b022-a8cfed55167c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:49.387196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67427a38-a59f-4cb2-b022-a8cfed55167c admin admin] PATH_INFO: `/v3/projects/57665a3a023246b383edb8d9255610c2/users/d384543810ff4470877ed01745831502/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:49.433005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67427a38-a59f-4cb2-b022-a8cfed55167c admin admin] RBAC: Authorizing `identity:create_grant(project_id=57665a3a023246b383edb8d9255610c2, user_id=d384543810ff4470877ed01745831502, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:49.444724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67427a38-a59f-4cb2-b022-a8cfed55167c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:49.573299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 863/1716] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:49 2022] PUT /identity/v3/projects/57665a3a023246b383edb8d9255610c2/users/d384543810ff4470877ed01745831502/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:49.588407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b337e77-532d-4461-a017-8ed517f4145e None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:49.588407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b337e77-532d-4461-a017-8ed517f4145e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:49.591657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b337e77-532d-4461-a017-8ed517f4145e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:49.644496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3b337e77-532d-4461-a017-8ed517f4145e None None] MFA Rules not processed for user `d384543810ff4470877ed01745831502`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:49.645128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3b337e77-532d-4461-a017-8ed517f4145e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:49.735335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:49.765180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 854/1717] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:04:49.773978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67427a38-a59f-4cb2-b022-a8cfed55167c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:50.075289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67427a38-a59f-4cb2-b022-a8cfed55167c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:50.099217 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34b10b-2246-43f6-aea1-c610400e688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:50.102150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34b10b-2246-43f6-aea1-c610400e688e admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:50.102680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34b10b-2246-43f6-aea1-c610400e688e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:50.103047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34b10b-2246-43f6-aea1-c610400e688e admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:50.104195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34b10b-2246-43f6-aea1-c610400e688e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:50.113385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34b10b-2246-43f6-aea1-c610400e688e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:50.147825 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:50.352206 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 864/1718] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:49 2022] POST /identity/v3/projects => generated 393 bytes in 579 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:50.353331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b337e77-532d-4461-a017-8ed517f4145e None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:50.398900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b337e77-532d-4461-a017-8ed517f4145e None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:50.443936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34b10b-2246-43f6-aea1-c610400e688e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:50.446658 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-dd34b10b-2246-43f6-aea1-c610400e688e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:50.647235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34b10b-2246-43f6-aea1-c610400e688e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:50.675123 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:50.678392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:50.682098 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:50.683089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] PATH_INFO: `/v3/projects/5cee2cf82c01477db791e979a1a8e3ac` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:50.689150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cee2cf82c01477db791e979a1a8e3ac)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:50.699769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:50.743141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d0cdd-6aa0-4fe8-a986-4231cf9b3a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:50.744881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d0cdd-6aa0-4fe8-a986-4231cf9b3a66 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:50.747280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d0cdd-6aa0-4fe8-a986-4231cf9b3a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:50.748199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d0cdd-6aa0-4fe8-a986-4231cf9b3a66 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:50.752762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d0cdd-6aa0-4fe8-a986-4231cf9b3a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:50.825198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d0cdd-6aa0-4fe8-a986-4231cf9b3a66 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:51.074679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 855/1719] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:50 2022] POST /identity/v3/users => generated 303 bytes in 721 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:51.079976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d0cdd-6aa0-4fe8-a986-4231cf9b3a66 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:51.095635 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cee2cf82c01477db791e979a1a8e3ac', 'request_id': 'req-5612b972-a02f-48f5-9bef-2f225ae6504b'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:51.116327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cee2cf82c01477db791e979a1a8e3ac', 'request_id': 'req-5612b972-a02f-48f5-9bef-2f225ae6504b'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:51.127821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cee2cf82c01477db791e979a1a8e3ac', 'request_id': 'req-5612b972-a02f-48f5-9bef-2f225ae6504b'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:51.144426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d0cdd-6aa0-4fe8-a986-4231cf9b3a66 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:51.311887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] The token cache is being invalidate because project 5cee2cf82c01477db791e979a1a8e3ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:04:51.312412 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:51.327466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:51.342557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 865/1720] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:04:50 2022] DELETE /identity/v3/projects/5cee2cf82c01477db791e979a1a8e3ac => generated 0 bytes in 899 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:04:51.344050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:51.373559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:51.513254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affecc7a-78d3-436c-b81d-03238952bfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:51.516437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affecc7a-78d3-436c-b81d-03238952bfa2 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:51.517184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affecc7a-78d3-436c-b81d-03238952bfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:51.517915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affecc7a-78d3-436c-b81d-03238952bfa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:51.521327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affecc7a-78d3-436c-b81d-03238952bfa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:51.525947 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affecc7a-78d3-436c-b81d-03238952bfa2 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:51.596830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 856/1721] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 521 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:51.599687 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8d29c-20c2-4afb-8559-cb1470c8d627 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:51.600118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8d29c-20c2-4afb-8559-cb1470c8d627 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:51.600512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8d29c-20c2-4afb-8559-cb1470c8d627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:51.659923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-83c8d29c-20c2-4afb-8559-cb1470c8d627 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:51.666441 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-83c8d29c-20c2-4afb-8559-cb1470c8d627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:51.707689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 857/1722] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:51.715679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8d29c-20c2-4afb-8559-cb1470c8d627 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:51.724130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-83c8d29c-20c2-4afb-8559-cb1470c8d627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:51.772775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8d29c-20c2-4afb-8559-cb1470c8d627 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:51.790633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b102b0-f9ae-4f59-b79a-7ca61ecbcb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:51.792116 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b102b0-f9ae-4f59-b79a-7ca61ecbcb7f admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:51.794366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b102b0-f9ae-4f59-b79a-7ca61ecbcb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:51.795274 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b102b0-f9ae-4f59-b79a-7ca61ecbcb7f admin admin] PATH_INFO: `/v3/projects/721c63a11fe3470eb30eef8363826b5c/users/afbf40a25f464bbda21afb71ce247044/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:51.812565 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612b972-a02f-48f5-9bef-2f225ae6504b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:51.832560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:51.835597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:51.838749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:51.839452 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] PATH_INFO: `/v3/projects/3726327cbac64e5a86392b6d2ae2cf4b` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:51.842622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3726327cbac64e5a86392b6d2ae2cf4b)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:51.848878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:51.871309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b102b0-f9ae-4f59-b79a-7ca61ecbcb7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=721c63a11fe3470eb30eef8363826b5c, user_id=afbf40a25f464bbda21afb71ce247044, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:51.874633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b102b0-f9ae-4f59-b79a-7ca61ecbcb7f admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:52.024371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 858/1723] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:51 2022] PUT /identity/v3/projects/721c63a11fe3470eb30eef8363826b5c/users/afbf40a25f464bbda21afb71ce247044/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:52.032904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff38704-e1de-4ad1-ac00-c4ee5021ca79 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:52.042072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff38704-e1de-4ad1-ac00-c4ee5021ca79 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:52.046602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff38704-e1de-4ad1-ac00-c4ee5021ca79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:52.056310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:52.127802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-aff38704-e1de-4ad1-ac00-c4ee5021ca79 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:52.137106 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-aff38704-e1de-4ad1-ac00-c4ee5021ca79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:52.257851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3726327cbac64e5a86392b6d2ae2cf4b', 'request_id': 'req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:52.280231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3726327cbac64e5a86392b6d2ae2cf4b', 'request_id': 'req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:52.303630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3726327cbac64e5a86392b6d2ae2cf4b', 'request_id': 'req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:52.332589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:52.375428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 859/1724] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:52.378684 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574827e-39a3-4f93-a77d-6ba1ff07dab2 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:52.380506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574827e-39a3-4f93-a77d-6ba1ff07dab2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:52.381562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574827e-39a3-4f93-a77d-6ba1ff07dab2 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:52.386268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 860/1725] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 19:04:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:04:52.388016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574827e-39a3-4f93-a77d-6ba1ff07dab2 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:52.610994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] The token cache is being invalidate because project 3726327cbac64e5a86392b6d2ae2cf4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:04:52.612498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:52.614103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:04:52.629804 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 866/1726] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:04:51 2022] DELETE /identity/v3/projects/3726327cbac64e5a86392b6d2ae2cf4b => generated 0 bytes in 1286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:52.631907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:52.634717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:52.674002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574827e-39a3-4f93-a77d-6ba1ff07dab2 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:52.688357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc75e9a-e146-4efb-a892-6ad6c26d27ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:52.689515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc75e9a-e146-4efb-a892-6ad6c26d27ef admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:52.691592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc75e9a-e146-4efb-a892-6ad6c26d27ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:52.694629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f182a1-5b77-43a8-8f01-ccd9463a5d6e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:52.695165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc75e9a-e146-4efb-a892-6ad6c26d27ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:52.696854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc75e9a-e146-4efb-a892-6ad6c26d27ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:52.704329 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f867e85-85ab-4e0f-a638-68695dafe0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:52.705665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f867e85-85ab-4e0f-a638-68695dafe0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:52.706762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc75e9a-e146-4efb-a892-6ad6c26d27ef admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:52.725936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f867e85-85ab-4e0f-a638-68695dafe0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:52.727347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f867e85-85ab-4e0f-a638-68695dafe0d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:52.731433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f867e85-85ab-4e0f-a638-68695dafe0d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:04:52.733599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f867e85-85ab-4e0f-a638-68695dafe0d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:52.738519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f867e85-85ab-4e0f-a638-68695dafe0d7 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:52.779658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 861/1727] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:52.792987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 867/1728] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:04:52.797878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d4940-70d3-4275-a881-c431c2c755c8 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:52.798629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d4940-70d3-4275-a881-c431c2c755c8 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:52.799216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d4940-70d3-4275-a881-c431c2c755c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:52.825155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12bf27-8998-4b0b-b5d5-afa1a1fec59f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:52.827306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12bf27-8998-4b0b-b5d5-afa1a1fec59f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:52.828471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12bf27-8998-4b0b-b5d5-afa1a1fec59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:52.892772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-293d4940-70d3-4275-a881-c431c2c755c8 None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:52.894179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-293d4940-70d3-4275-a881-c431c2c755c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:52.997439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-5a12bf27-8998-4b0b-b5d5-afa1a1fec59f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:53.014870 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5a12bf27-8998-4b0b-b5d5-afa1a1fec59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:53.074035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 862/1729] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:53.077092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12bf27-8998-4b0b-b5d5-afa1a1fec59f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:53.087371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5a12bf27-8998-4b0b-b5d5-afa1a1fec59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:53.164087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12bf27-8998-4b0b-b5d5-afa1a1fec59f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:53.238970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf19f9-5ca4-4a4c-976c-d2d490b1733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:53.249007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf19f9-5ca4-4a4c-976c-d2d490b1733c admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:53.252514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf19f9-5ca4-4a4c-976c-d2d490b1733c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:53.254168 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf19f9-5ca4-4a4c-976c-d2d490b1733c admin admin] PATH_INFO: `/v3/projects/721c63a11fe3470eb30eef8363826b5c/users/afbf40a25f464bbda21afb71ce247044/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:53.264760 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf19f9-5ca4-4a4c-976c-d2d490b1733c admin admin] RBAC: Authorizing `identity:create_grant(project_id=721c63a11fe3470eb30eef8363826b5c, user_id=afbf40a25f464bbda21afb71ce247044, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:53.270534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 868/1730] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 19:04:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:53.271472 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d4940-70d3-4275-a881-c431c2c755c8 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:53.280623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf19f9-5ca4-4a4c-976c-d2d490b1733c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:53.292946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d4940-70d3-4275-a881-c431c2c755c8 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:53.316275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c811e-e6ee-4de5-8c53-e02c5dc7b8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:53.317462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c811e-e6ee-4de5-8c53-e02c5dc7b8c1 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:53.320159 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c811e-e6ee-4de5-8c53-e02c5dc7b8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:53.322102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c811e-e6ee-4de5-8c53-e02c5dc7b8c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:53.323076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c811e-e6ee-4de5-8c53-e02c5dc7b8c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:53.336599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c811e-e6ee-4de5-8c53-e02c5dc7b8c1 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:53.345315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 863/1731] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:53 2022] PUT /identity/v3/projects/721c63a11fe3470eb30eef8363826b5c/users/afbf40a25f464bbda21afb71ce247044/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:53.350292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0df11f-2a5a-4d83-8df9-fc028e35ce0f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:53.351429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0df11f-2a5a-4d83-8df9-fc028e35ce0f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:53.352471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0df11f-2a5a-4d83-8df9-fc028e35ce0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:53.437122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-4c0df11f-2a5a-4d83-8df9-fc028e35ce0f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:53.443321 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4c0df11f-2a5a-4d83-8df9-fc028e35ce0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:53.546394 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 869/1732] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:53 2022] POST /identity/v3/projects => generated 387 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:53.552242 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c982f6a-300a-4172-8b13-bb5bcee0d92c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:53.552242 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c982f6a-300a-4172-8b13-bb5bcee0d92c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:53.555947 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c982f6a-300a-4172-8b13-bb5bcee0d92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:53.583309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 864/1733] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:53.584992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0df11f-2a5a-4d83-8df9-fc028e35ce0f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:53.604872 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0df11f-2a5a-4d83-8df9-fc028e35ce0f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:53.733771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-6c982f6a-300a-4172-8b13-bb5bcee0d92c None None] MFA Rules not processed for user `afbf40a25f464bbda21afb71ce247044`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:53.744207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6c982f6a-300a-4172-8b13-bb5bcee0d92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:53.897533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a382c-3e9e-42cc-94ac-a4bcd65562fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:53.901933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a382c-3e9e-42cc-94ac-a4bcd65562fc admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:53.901933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a382c-3e9e-42cc-94ac-a4bcd65562fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:53.901933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a382c-3e9e-42cc-94ac-a4bcd65562fc admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:53.905968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a382c-3e9e-42cc-94ac-a4bcd65562fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:53.910470 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a382c-3e9e-42cc-94ac-a4bcd65562fc admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:54.024487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 870/1734] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:54.025894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c982f6a-300a-4172-8b13-bb5bcee0d92c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:54.056003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c982f6a-300a-4172-8b13-bb5bcee0d92c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:54.058214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 865/1735] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:53 2022] POST /identity/v3/users => generated 294 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:54.059743 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a382c-3e9e-42cc-94ac-a4bcd65562fc admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:54.064297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-840a382c-3e9e-42cc-94ac-a4bcd65562fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:54.076202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821dc2-4d3d-42fc-8339-7e2024d6d345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:54.082742 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821dc2-4d3d-42fc-8339-7e2024d6d345 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:54.082742 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821dc2-4d3d-42fc-8339-7e2024d6d345 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:54.082742 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821dc2-4d3d-42fc-8339-7e2024d6d345 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:54.082742 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7821dc2-4d3d-42fc-8339-7e2024d6d345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:04:54.082742 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7821dc2-4d3d-42fc-8339-7e2024d6d345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:54.082742 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7821dc2-4d3d-42fc-8339-7e2024d6d345 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:54.125270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 871/1736] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:04:54.125270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821dc2-4d3d-42fc-8339-7e2024d6d345 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:54.142667 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821dc2-4d3d-42fc-8339-7e2024d6d345 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:54.161923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de4158-4eab-401c-83e4-a904c658916d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:54.165398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de4158-4eab-401c-83e4-a904c658916d admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:54.165703 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de4158-4eab-401c-83e4-a904c658916d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:54.169891 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de4158-4eab-401c-83e4-a904c658916d admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:54.174857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de4158-4eab-401c-83e4-a904c658916d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:54.180585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de4158-4eab-401c-83e4-a904c658916d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:54.226883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 872/1737] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:54.232252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de4158-4eab-401c-83e4-a904c658916d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:54.233087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a382c-3e9e-42cc-94ac-a4bcd65562fc admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:54.256014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f45aa-94d4-4715-af3f-ff7af1f172b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:54.262898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f45aa-94d4-4715-af3f-ff7af1f172b8 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:54.266838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f45aa-94d4-4715-af3f-ff7af1f172b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:54.266838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f45aa-94d4-4715-af3f-ff7af1f172b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:54.290428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de4158-4eab-401c-83e4-a904c658916d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:54.308310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec9190-0e6d-4352-9b0d-a944c84b4b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:54.309432 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec9190-0e6d-4352-9b0d-a944c84b4b04 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:54.310104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec9190-0e6d-4352-9b0d-a944c84b4b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:54.310837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec9190-0e6d-4352-9b0d-a944c84b4b04 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:54.315581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec9190-0e6d-4352-9b0d-a944c84b4b04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:54.326500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec9190-0e6d-4352-9b0d-a944c84b4b04 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:54.328805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f45aa-94d4-4715-af3f-ff7af1f172b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:54.331941 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f45aa-94d4-4715-af3f-ff7af1f172b8 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:54.361028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 866/1738] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:54.381289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f45aa-94d4-4715-af3f-ff7af1f172b8 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:54.443300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f45aa-94d4-4715-af3f-ff7af1f172b8 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:54.482726 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00779c-7439-4b53-b766-efc500d0d128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:54.485766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00779c-7439-4b53-b766-efc500d0d128 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:54.486512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00779c-7439-4b53-b766-efc500d0d128 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:54.488362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00779c-7439-4b53-b766-efc500d0d128 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:54.491403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00779c-7439-4b53-b766-efc500d0d128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:54.496479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00779c-7439-4b53-b766-efc500d0d128 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:54.590773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 867/1739] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:54.593330 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461630d6-ed2a-4f09-af64-56ba34607ff0 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:54.598490 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461630d6-ed2a-4f09-af64-56ba34607ff0 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:54.599170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461630d6-ed2a-4f09-af64-56ba34607ff0 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:54.619196 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 868/1740] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 19:04:54 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:04:54.619196 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461630d6-ed2a-4f09-af64-56ba34607ff0 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:54.678551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461630d6-ed2a-4f09-af64-56ba34607ff0 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:54.693457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a2dd8-31c1-454b-865f-02d974ba7e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:54.694940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a2dd8-31c1-454b-865f-02d974ba7e79 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:54.695423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a2dd8-31c1-454b-865f-02d974ba7e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:54.695904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a2dd8-31c1-454b-865f-02d974ba7e79 admin admin] PATH_INFO: `/v3/projects/bda09e8a03574c47babb22875ac15c6f/users/3adbec6af2c04d2586a2c0470afe9add/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:54.730412 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a2dd8-31c1-454b-865f-02d974ba7e79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda09e8a03574c47babb22875ac15c6f, user_id=3adbec6af2c04d2586a2c0470afe9add, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:54.736404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a2dd8-31c1-454b-865f-02d974ba7e79 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:54.742490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 873/1741] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:54 2022] POST /identity/v3/projects => generated 381 bytes in 514 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:54.744192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec9190-0e6d-4352-9b0d-a944c84b4b04 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:54.747564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c3ec9190-0e6d-4352-9b0d-a944c84b4b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:54.780536 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:54.899108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 869/1742] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:54 2022] PUT /identity/v3/projects/bda09e8a03574c47babb22875ac15c6f/users/3adbec6af2c04d2586a2c0470afe9add/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:54.904173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498de97f-7ec4-4845-90c3-255cb06070a7 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:54.904565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498de97f-7ec4-4845-90c3-255cb06070a7 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:54.904858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498de97f-7ec4-4845-90c3-255cb06070a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:54.943447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-498de97f-7ec4-4845-90c3-255cb06070a7 None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:54.947636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-498de97f-7ec4-4845-90c3-255cb06070a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:55.016889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:55.123861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec9190-0e6d-4352-9b0d-a944c84b4b04 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:55.185280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 870/1743] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 19:04:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:55.188185 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498de97f-7ec4-4845-90c3-255cb06070a7 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:55.352687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5915882-0ea7-488e-83b8-bc9b24e58907 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:55.361452 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5915882-0ea7-488e-83b8-bc9b24e58907 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:55.362011 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5915882-0ea7-488e-83b8-bc9b24e58907 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:55.362569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5915882-0ea7-488e-83b8-bc9b24e58907 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:55.409180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498de97f-7ec4-4845-90c3-255cb06070a7 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:55.421035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5915882-0ea7-488e-83b8-bc9b24e58907 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:55.425279 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5915882-0ea7-488e-83b8-bc9b24e58907 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:55.425832 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730bb12-bbff-447e-8661-fce219287c97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:55.426567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730bb12-bbff-447e-8661-fce219287c97 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:55.426868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730bb12-bbff-447e-8661-fce219287c97 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:55.427221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730bb12-bbff-447e-8661-fce219287c97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:55.432955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5730bb12-bbff-447e-8661-fce219287c97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:55.464663 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 874/1744] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:55.466095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5915882-0ea7-488e-83b8-bc9b24e58907 service nova] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:55.699384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5915882-0ea7-488e-83b8-bc9b24e58907 service nova] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:55.700256 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730bb12-bbff-447e-8661-fce219287c97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:55.705801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730bb12-bbff-447e-8661-fce219287c97 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:55.719596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67052f6-be1a-438f-8ae7-37c72d62c40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:55.720604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67052f6-be1a-438f-8ae7-37c72d62c40f admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:55.720905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67052f6-be1a-438f-8ae7-37c72d62c40f admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:55.721653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67052f6-be1a-438f-8ae7-37c72d62c40f admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:55.724175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67052f6-be1a-438f-8ae7-37c72d62c40f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:55.733900 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67052f6-be1a-438f-8ae7-37c72d62c40f admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:55.744772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 871/1745] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:55.749274 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547943d-9c45-4f3f-8425-957611f909bc None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:55.750577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547943d-9c45-4f3f-8425-957611f909bc None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:55.750944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547943d-9c45-4f3f-8425-957611f909bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:55.842620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-1547943d-9c45-4f3f-8425-957611f909bc None None] MFA Rules not processed for user `3adbec6af2c04d2586a2c0470afe9add`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:55.843846 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1547943d-9c45-4f3f-8425-957611f909bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:55.897526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 875/1746] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:55 2022] POST /identity/v3/users => generated 291 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:55.902016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67052f6-be1a-438f-8ae7-37c72d62c40f admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:55.971074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67052f6-be1a-438f-8ae7-37c72d62c40f admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:55.997663 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b0084-18ad-4525-9ca7-5fc26548cece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:55.998564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b0084-18ad-4525-9ca7-5fc26548cece admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:55.998915 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b0084-18ad-4525-9ca7-5fc26548cece admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:56.001537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b0084-18ad-4525-9ca7-5fc26548cece admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:56.003958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b0084-18ad-4525-9ca7-5fc26548cece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:56.010561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b0084-18ad-4525-9ca7-5fc26548cece admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:56.042409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 876/1747] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:56.044265 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b0084-18ad-4525-9ca7-5fc26548cece admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:56.107601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b0084-18ad-4525-9ca7-5fc26548cece admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:56.130595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aae896-42fa-4910-b1c1-894b2a1944ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:56.131812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aae896-42fa-4910-b1c1-894b2a1944ef admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:56.132249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aae896-42fa-4910-b1c1-894b2a1944ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:56.135862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aae896-42fa-4910-b1c1-894b2a1944ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:56.137016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aae896-42fa-4910-b1c1-894b2a1944ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:56.145089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aae896-42fa-4910-b1c1-894b2a1944ef admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:56.206536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 877/1748] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:56.216518 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aae896-42fa-4910-b1c1-894b2a1944ef admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:56.253885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aae896-42fa-4910-b1c1-894b2a1944ef admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:56.279512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 872/1749] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 534 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:56.294437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547943d-9c45-4f3f-8425-957611f909bc None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:56.306409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbda606-5f5e-48d1-b3dc-f7f82512223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:56.307991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbda606-5f5e-48d1-b3dc-f7f82512223d admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:56.308572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbda606-5f5e-48d1-b3dc-f7f82512223d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:56.309119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbda606-5f5e-48d1-b3dc-f7f82512223d admin admin] PATH_INFO: `/v3/projects/eeab432513684a33911fe88335c88984/users/cb1d01d58303434fb81e185d72a09710/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:56.330236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547943d-9c45-4f3f-8425-957611f909bc None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:56.388022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce06c62-2198-4a7b-9688-bcb4b8ab6073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:56.389766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce06c62-2198-4a7b-9688-bcb4b8ab6073 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:56.395965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce06c62-2198-4a7b-9688-bcb4b8ab6073 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:56.395965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce06c62-2198-4a7b-9688-bcb4b8ab6073 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:56.402452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce06c62-2198-4a7b-9688-bcb4b8ab6073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:56.405377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbda606-5f5e-48d1-b3dc-f7f82512223d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeab432513684a33911fe88335c88984, user_id=cb1d01d58303434fb81e185d72a09710, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:56.410290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbda606-5f5e-48d1-b3dc-f7f82512223d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:56.415201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce06c62-2198-4a7b-9688-bcb4b8ab6073 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:56.599088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 873/1750] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:56 2022] POST /identity/v3/projects => generated 385 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:56.600158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 878/1751] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:56 2022] PUT /identity/v3/projects/eeab432513684a33911fe88335c88984/users/cb1d01d58303434fb81e185d72a09710/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:56.600432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce06c62-2198-4a7b-9688-bcb4b8ab6073 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:56.626131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb965630-ad2d-4672-91dd-f99853f9c24f None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:56.642637 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb965630-ad2d-4672-91dd-f99853f9c24f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:56.642637 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb965630-ad2d-4672-91dd-f99853f9c24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:56.733252 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce06c62-2198-4a7b-9688-bcb4b8ab6073 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:56.766588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-eb965630-ad2d-4672-91dd-f99853f9c24f None None] MFA Rules not processed for user `cb1d01d58303434fb81e185d72a09710`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:56.766805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44c365-06c7-463a-b3e1-13266a7238c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:56.766805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44c365-06c7-463a-b3e1-13266a7238c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:56.766805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44c365-06c7-463a-b3e1-13266a7238c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:56.766805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44c365-06c7-463a-b3e1-13266a7238c2 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:56.767293 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-eb965630-ad2d-4672-91dd-f99853f9c24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:56.767405 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44c365-06c7-463a-b3e1-13266a7238c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:56.767405 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44c365-06c7-463a-b3e1-13266a7238c2 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:56.936522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 879/1752] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:56.943675 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb965630-ad2d-4672-91dd-f99853f9c24f None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:56.962874 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb965630-ad2d-4672-91dd-f99853f9c24f None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:56.972255 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7b8ec-50c0-472e-a7f5-bf7f2b9cd88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:56.974310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7b8ec-50c0-472e-a7f5-bf7f2b9cd88b admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:56.976045 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7b8ec-50c0-472e-a7f5-bf7f2b9cd88b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:56.976045 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7b8ec-50c0-472e-a7f5-bf7f2b9cd88b admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:56.976621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7b8ec-50c0-472e-a7f5-bf7f2b9cd88b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:56.978827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7b8ec-50c0-472e-a7f5-bf7f2b9cd88b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:56.984077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 874/1753] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:56 2022] POST /identity/v3/users => generated 299 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:56.992598 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44c365-06c7-463a-b3e1-13266a7238c2 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:57.059942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44c365-06c7-463a-b3e1-13266a7238c2 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:57.076979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0307775-f644-4f3f-8fe7-94978fa65aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:57.079477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0307775-f644-4f3f-8fe7-94978fa65aed admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:57.080192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0307775-f644-4f3f-8fe7-94978fa65aed admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:57.081326 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0307775-f644-4f3f-8fe7-94978fa65aed admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:57.082634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0307775-f644-4f3f-8fe7-94978fa65aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:57.086317 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0307775-f644-4f3f-8fe7-94978fa65aed admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:57.113380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 880/1754] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:04:56 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:57.121256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7b8ec-50c0-472e-a7f5-bf7f2b9cd88b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:57.148222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 875/1755] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:57.149824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0307775-f644-4f3f-8fe7-94978fa65aed admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:57.208150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0307775-f644-4f3f-8fe7-94978fa65aed admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:57.409022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e45c1c-77d1-47c1-b581-b64b5e994eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:57.410855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e45c1c-77d1-47c1-b581-b64b5e994eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:57.411486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e45c1c-77d1-47c1-b581-b64b5e994eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:57.411987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e45c1c-77d1-47c1-b581-b64b5e994eaa admin admin] PATH_INFO: `/v3/projects/dd56f5edc3634ae1b60cf6dd26a61c63/users/a9415d4957c646df83a2c6eda1079238/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:57.412518 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7b8ec-50c0-472e-a7f5-bf7f2b9cd88b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:57.439015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e45c1c-77d1-47c1-b581-b64b5e994eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd56f5edc3634ae1b60cf6dd26a61c63, user_id=a9415d4957c646df83a2c6eda1079238, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:57.440381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d12b3f-5a0f-4166-91ce-d54addbc2de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:57.443988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e45c1c-77d1-47c1-b581-b64b5e994eaa admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:57.448415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d12b3f-5a0f-4166-91ce-d54addbc2de4 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:57.449030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d12b3f-5a0f-4166-91ce-d54addbc2de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:57.449501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d12b3f-5a0f-4166-91ce-d54addbc2de4 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:57.464292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d12b3f-5a0f-4166-91ce-d54addbc2de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:57.469064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d12b3f-5a0f-4166-91ce-d54addbc2de4 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:57.748479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 881/1756] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:04:57 2022] POST /identity/v3/users => generated 297 bytes in 631 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:04:57.754073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d12b3f-5a0f-4166-91ce-d54addbc2de4 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:57.754589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 876/1757] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:57 2022] PUT /identity/v3/projects/dd56f5edc3634ae1b60cf6dd26a61c63/users/a9415d4957c646df83a2c6eda1079238/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:57.756426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e45c1c-77d1-47c1-b581-b64b5e994eaa admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:57.918928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e45c1c-77d1-47c1-b581-b64b5e994eaa admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:57.930894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388dc0c-9025-448c-b7d9-a13ce0d4e355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:57.930894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388dc0c-9025-448c-b7d9-a13ce0d4e355 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:57.930894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388dc0c-9025-448c-b7d9-a13ce0d4e355 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:57.930894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388dc0c-9025-448c-b7d9-a13ce0d4e355 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:57.931549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388dc0c-9025-448c-b7d9-a13ce0d4e355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:57.933038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388dc0c-9025-448c-b7d9-a13ce0d4e355 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:57.950789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 877/1758] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:57.952705 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388dc0c-9025-448c-b7d9-a13ce0d4e355 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:57.970521 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388dc0c-9025-448c-b7d9-a13ce0d4e355 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:57.972740 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d12b3f-5a0f-4166-91ce-d54addbc2de4 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:57.999108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add709f7-9f58-408d-94c6-cbec8e731120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:58.000262 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add709f7-9f58-408d-94c6-cbec8e731120 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:58.000815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add709f7-9f58-408d-94c6-cbec8e731120 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:58.002387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add709f7-9f58-408d-94c6-cbec8e731120 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:58.004513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add709f7-9f58-408d-94c6-cbec8e731120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:58.009928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add709f7-9f58-408d-94c6-cbec8e731120 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:58.037730 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 878/1759] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:58.037730 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add709f7-9f58-408d-94c6-cbec8e731120 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:58.058638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add709f7-9f58-408d-94c6-cbec8e731120 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:58.071631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a11b4-2ffb-480d-8d5b-476e7dec6e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:58.088046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a11b4-2ffb-480d-8d5b-476e7dec6e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:58.088046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a11b4-2ffb-480d-8d5b-476e7dec6e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:58.088046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a11b4-2ffb-480d-8d5b-476e7dec6e10 admin admin] PATH_INFO: `/v3/projects/dd56f5edc3634ae1b60cf6dd26a61c63/users/a9415d4957c646df83a2c6eda1079238/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:58.088046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a11b4-2ffb-480d-8d5b-476e7dec6e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd56f5edc3634ae1b60cf6dd26a61c63, user_id=a9415d4957c646df83a2c6eda1079238, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:58.088046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a11b4-2ffb-480d-8d5b-476e7dec6e10 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:58.133557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 879/1760] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:58 2022] PUT /identity/v3/projects/dd56f5edc3634ae1b60cf6dd26a61c63/users/a9415d4957c646df83a2c6eda1079238/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:58.134584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a11b4-2ffb-480d-8d5b-476e7dec6e10 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:58.239604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba091d-7514-419f-9a7e-0ab788bd6956 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:58.240922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba091d-7514-419f-9a7e-0ab788bd6956 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:58.241976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba091d-7514-419f-9a7e-0ab788bd6956 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:58.242167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba091d-7514-419f-9a7e-0ab788bd6956 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:58.257576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba091d-7514-419f-9a7e-0ab788bd6956 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:58.262591 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba091d-7514-419f-9a7e-0ab788bd6956 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:58.288769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 882/1761] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:58.291309 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6d780-262a-4d10-b1a0-3d846cff2f36 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:58.291931 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6d780-262a-4d10-b1a0-3d846cff2f36 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:58.292222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6d780-262a-4d10-b1a0-3d846cff2f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:58.334825 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a11b4-2ffb-480d-8d5b-476e7dec6e10 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:58.350803 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-dbd6d780-262a-4d10-b1a0-3d846cff2f36 None None] MFA Rules not processed for user `a9415d4957c646df83a2c6eda1079238`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:58.352221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42bfd3-d279-4cfa-9edf-daeea0c75076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:58.352625 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-dbd6d780-262a-4d10-b1a0-3d846cff2f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:58.353251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42bfd3-d279-4cfa-9edf-daeea0c75076 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:58.353549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42bfd3-d279-4cfa-9edf-daeea0c75076 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:58.353835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42bfd3-d279-4cfa-9edf-daeea0c75076 admin admin] PATH_INFO: `/v3/projects/2330c81ef5f646b58767c187eb16b660/users/bc5b75f0f6724b4e94f646b2eaeb46fa/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:58.482888 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:04:58.490451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42bfd3-d279-4cfa-9edf-daeea0c75076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2330c81ef5f646b58767c187eb16b660, user_id=bc5b75f0f6724b4e94f646b2eaeb46fa, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:58.493541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42bfd3-d279-4cfa-9edf-daeea0c75076 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:58.569954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 883/1762] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:58.598617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6d780-262a-4d10-b1a0-3d846cff2f36 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:58.616777 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 880/1763] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:58 2022] PUT /identity/v3/projects/2330c81ef5f646b58767c187eb16b660/users/bc5b75f0f6724b4e94f646b2eaeb46fa/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:58.623200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42bfd3-d279-4cfa-9edf-daeea0c75076 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:58.834592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42bfd3-d279-4cfa-9edf-daeea0c75076 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:58.835335 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6d780-262a-4d10-b1a0-3d846cff2f36 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:58.847368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b40b9-b6d1-4aba-9967-b6f3842cc3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:58.848194 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b40b9-b6d1-4aba-9967-b6f3842cc3eb admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:58.848490 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b40b9-b6d1-4aba-9967-b6f3842cc3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:58.848816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b40b9-b6d1-4aba-9967-b6f3842cc3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:58.849672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b40b9-b6d1-4aba-9967-b6f3842cc3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:58.852426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b40b9-b6d1-4aba-9967-b6f3842cc3eb admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:58.903922 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 881/1764] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:04:58.905333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b40b9-b6d1-4aba-9967-b6f3842cc3eb admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:58.937127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b40b9-b6d1-4aba-9967-b6f3842cc3eb admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:58.974235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873fa3a-74f3-4405-864e-fa27fad16c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:58.975821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873fa3a-74f3-4405-864e-fa27fad16c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:58.975946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873fa3a-74f3-4405-864e-fa27fad16c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:58.976192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873fa3a-74f3-4405-864e-fa27fad16c55 admin admin] PATH_INFO: `/v3/projects/2330c81ef5f646b58767c187eb16b660/users/bc5b75f0f6724b4e94f646b2eaeb46fa/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:58.981320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873fa3a-74f3-4405-864e-fa27fad16c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2330c81ef5f646b58767c187eb16b660, user_id=bc5b75f0f6724b4e94f646b2eaeb46fa, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:58.988022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873fa3a-74f3-4405-864e-fa27fad16c55 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:59.041018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 882/1765] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:04:58 2022] PUT /identity/v3/projects/2330c81ef5f646b58767c187eb16b660/users/bc5b75f0f6724b4e94f646b2eaeb46fa/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:04:59.056405 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e59ee-d837-4add-90c4-acf52194fe9d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:59.056975 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e59ee-d837-4add-90c4-acf52194fe9d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:59.057514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e59ee-d837-4add-90c4-acf52194fe9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:59.070632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3073256-300c-4188-ab1a-bac303afe65c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:59.071550 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3073256-300c-4188-ab1a-bac303afe65c service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:59.072131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3073256-300c-4188-ab1a-bac303afe65c service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:59.072469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3073256-300c-4188-ab1a-bac303afe65c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:59.167589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-d50e59ee-d837-4add-90c4-acf52194fe9d None None] MFA Rules not processed for user `bc5b75f0f6724b4e94f646b2eaeb46fa`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:04:59.169841 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d50e59ee-d837-4add-90c4-acf52194fe9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:59.355176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 883/1766] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:04:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:04:59.400777 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e59ee-d837-4add-90c4-acf52194fe9d None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:59.421386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d50e59ee-d837-4add-90c4-acf52194fe9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:04:59.457314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3073256-300c-4188-ab1a-bac303afe65c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:59.460617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3073256-300c-4188-ab1a-bac303afe65c service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:59.506267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 884/1767] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 907 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:04:59.556179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3073256-300c-4188-ab1a-bac303afe65c service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:59.572831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3073256-300c-4188-ab1a-bac303afe65c service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:59.580870 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e59ee-d837-4add-90c4-acf52194fe9d None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:59.588337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76729ee-f892-488e-a141-63922c48e8c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:59.589296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76729ee-f892-488e-a141-63922c48e8c7 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:59.589570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76729ee-f892-488e-a141-63922c48e8c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:59.589891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76729ee-f892-488e-a141-63922c48e8c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:59.614406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76729ee-f892-488e-a141-63922c48e8c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:59.616394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76729ee-f892-488e-a141-63922c48e8c7 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:59.638853 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 884/1768] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:04:59.835513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:59.836913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:59.837261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:59.837541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] PATH_INFO: `/v3/users/3adbec6af2c04d2586a2c0470afe9add` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:04:59.846868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3adbec6af2c04d2586a2c0470afe9add)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:04:59.849126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:04:59.957785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76729ee-f892-488e-a141-63922c48e8c7 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:04:59.973191 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76729ee-f892-488e-a141-63922c48e8c7 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:04:59.983989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae54b3b-e45a-483d-b897-006d1205d270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:04:59.985846 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae54b3b-e45a-483d-b897-006d1205d270 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:04:59.986631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae54b3b-e45a-483d-b897-006d1205d270 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:04:59.987298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae54b3b-e45a-483d-b897-006d1205d270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:00.146915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae54b3b-e45a-483d-b897-006d1205d270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:00.150563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae54b3b-e45a-483d-b897-006d1205d270 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:00.180556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 885/1769] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:04:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:05:00.183467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:00.219449 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3adbec6af2c04d2586a2c0470afe9add'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:00.220808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3adbec6af2c04d2586a2c0470afe9add'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:00.243061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3adbec6af2c04d2586a2c0470afe9add'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:00.278484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3adbec6af2c04d2586a2c0470afe9add'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:00.649638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3adbec6af2c04d2586a2c0470afe9add'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:00.667075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 885/1770] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:04:59 2022] DELETE /identity/v3/users/3adbec6af2c04d2586a2c0470afe9add => generated 0 bytes in 1108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:00.670530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa5df9-78b9-4aea-9181-0d7c989c98a7 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:00.671191 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa5df9-78b9-4aea-9181-0d7c989c98a7 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:00.671416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa5df9-78b9-4aea-9181-0d7c989c98a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:00.777934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-8dfa5df9-78b9-4aea-9181-0d7c989c98a7 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:00.805570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8dfa5df9-78b9-4aea-9181-0d7c989c98a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:01.220265 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 886/1771] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 551 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:01.310927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:01.313405 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:01.490826 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:01.496789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6acc6-87b9-4e10-8fb3-833bf6d3caf3 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:01.504361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5747bb-50bf-4e83-95e6-d83353d59282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:01.505272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5747bb-50bf-4e83-95e6-d83353d59282 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:01.506066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5747bb-50bf-4e83-95e6-d83353d59282 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:01.506569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5747bb-50bf-4e83-95e6-d83353d59282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:01.543805 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5747bb-50bf-4e83-95e6-d83353d59282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:01.549590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5747bb-50bf-4e83-95e6-d83353d59282 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:01.592934 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 886/1772] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:05:03.082481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:03.595860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:03.746704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa5df9-78b9-4aea-9181-0d7c989c98a7 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:03.748219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8dfa5df9-78b9-4aea-9181-0d7c989c98a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:03.768269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa5df9-78b9-4aea-9181-0d7c989c98a7 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:03.790579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:03.791829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:03.792388 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:03.792807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] PATH_INFO: `/v3/projects/bda09e8a03574c47babb22875ac15c6f` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:03.794674 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bda09e8a03574c47babb22875ac15c6f)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:03.797395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:03.983691 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bda09e8a03574c47babb22875ac15c6f', 'request_id': 'req-c9d63d74-297f-48d6-9d07-4e15f9aa033e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:04.074692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bda09e8a03574c47babb22875ac15c6f', 'request_id': 'req-c9d63d74-297f-48d6-9d07-4e15f9aa033e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:04.089907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bda09e8a03574c47babb22875ac15c6f', 'request_id': 'req-c9d63d74-297f-48d6-9d07-4e15f9aa033e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:04.106450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] The token cache is being invalidate because project bda09e8a03574c47babb22875ac15c6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:04.106951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:04.108037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:04.110739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 887/1773] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:03 2022] DELETE /identity/v3/projects/bda09e8a03574c47babb22875ac15c6f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:04.120321 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5747bb-50bf-4e83-95e6-d83353d59282 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:04.138133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5747bb-50bf-4e83-95e6-d83353d59282 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:04.311624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:04.314299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:04.314958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:04.315893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] PATH_INFO: `/v3/users/a9415d4957c646df83a2c6eda1079238` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:04.317999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9415d4957c646df83a2c6eda1079238)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:04.321881 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:04.368228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:04.771591 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9415d4957c646df83a2c6eda1079238'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:04.772318 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9415d4957c646df83a2c6eda1079238'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:04.782965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9415d4957c646df83a2c6eda1079238'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:04.803949 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9415d4957c646df83a2c6eda1079238'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:04.913169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9415d4957c646df83a2c6eda1079238'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:04.926666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 887/1774] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:04 2022] DELETE /identity/v3/users/a9415d4957c646df83a2c6eda1079238 => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:05.100949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:05.771786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:05.775320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:05.843672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d63d74-297f-48d6-9d07-4e15f9aa033e admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:05.992825 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:05.993954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:05.994252 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:05.994793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] PATH_INFO: `/v3/projects/dd56f5edc3634ae1b60cf6dd26a61c63` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:05.996914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd56f5edc3634ae1b60cf6dd26a61c63)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:05.998105 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:06.045181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:06.048810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:06.070822 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38a725-f837-46aa-8794-89dfc93b76da admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:06.077745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132c39e-246e-4e30-9fd9-9138a887a59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:06.078907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132c39e-246e-4e30-9fd9-9138a887a59f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:06.079595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132c39e-246e-4e30-9fd9-9138a887a59f admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:06.080032 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132c39e-246e-4e30-9fd9-9138a887a59f admin admin] PATH_INFO: `/v3/users/cb1d01d58303434fb81e185d72a09710` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:06.081798 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132c39e-246e-4e30-9fd9-9138a887a59f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb1d01d58303434fb81e185d72a09710)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:06.084180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132c39e-246e-4e30-9fd9-9138a887a59f admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:06.171511 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:06.179003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd56f5edc3634ae1b60cf6dd26a61c63', 'request_id': 'req-e53d95a8-a189-495f-aeda-2dd811081731'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.282939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd56f5edc3634ae1b60cf6dd26a61c63', 'request_id': 'req-e53d95a8-a189-495f-aeda-2dd811081731'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.294999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd56f5edc3634ae1b60cf6dd26a61c63', 'request_id': 'req-e53d95a8-a189-495f-aeda-2dd811081731'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.310179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] The token cache is being invalidate because project dd56f5edc3634ae1b60cf6dd26a61c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:06.312312 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.312312 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.314414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 888/1775] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:05 2022] DELETE /identity/v3/projects/dd56f5edc3634ae1b60cf6dd26a61c63 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:06.315784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:06.317534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:06.371413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1132c39e-246e-4e30-9fd9-9138a887a59f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb1d01d58303434fb81e185d72a09710'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.372812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1132c39e-246e-4e30-9fd9-9138a887a59f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb1d01d58303434fb81e185d72a09710'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.382817 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1132c39e-246e-4e30-9fd9-9138a887a59f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb1d01d58303434fb81e185d72a09710'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.398942 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1132c39e-246e-4e30-9fd9-9138a887a59f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb1d01d58303434fb81e185d72a09710'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.426276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d95a8-a189-495f-aeda-2dd811081731 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:06.433656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c161a-af55-4d25-9bd2-98f80d9306f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:06.434910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c161a-af55-4d25-9bd2-98f80d9306f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:06.435295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c161a-af55-4d25-9bd2-98f80d9306f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:06.435672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c161a-af55-4d25-9bd2-98f80d9306f4 admin admin] PATH_INFO: `/v3/users/d384543810ff4470877ed01745831502` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:06.438075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c161a-af55-4d25-9bd2-98f80d9306f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d384543810ff4470877ed01745831502)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:06.440785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c161a-af55-4d25-9bd2-98f80d9306f4 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:06.482669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1132c39e-246e-4e30-9fd9-9138a887a59f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb1d01d58303434fb81e185d72a09710'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.491983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 888/1776] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:06 2022] DELETE /identity/v3/users/cb1d01d58303434fb81e185d72a09710 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:06.493392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353db2-b62f-43af-ada8-f913c240bde1 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:06.493691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353db2-b62f-43af-ada8-f913c240bde1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:06.493977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353db2-b62f-43af-ada8-f913c240bde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:06.517956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-fe353db2-b62f-43af-ada8-f913c240bde1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:06.519350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fe353db2-b62f-43af-ada8-f913c240bde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:06.599492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 889/1777] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:06.603623 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38c6e7-6de6-4234-8a8e-f06c85c6debb None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:06.604087 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38c6e7-6de6-4234-8a8e-f06c85c6debb None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:06.604739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38c6e7-6de6-4234-8a8e-f06c85c6debb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:06.633614 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-fe38c6e7-6de6-4234-8a8e-f06c85c6debb None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:06.635113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fe38c6e7-6de6-4234-8a8e-f06c85c6debb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:06.644201 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 890/1778] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:06.645765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ca56a-9308-4e14-948d-c69c1bf7ebf8 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:06.646206 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ca56a-9308-4e14-948d-c69c1bf7ebf8 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:06.646735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ca56a-9308-4e14-948d-c69c1bf7ebf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:06.671622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-cd1ca56a-9308-4e14-948d-c69c1bf7ebf8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:06.673436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-cd1ca56a-9308-4e14-948d-c69c1bf7ebf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:06.682481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-fa4c161a-af55-4d25-9bd2-98f80d9306f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd384543810ff4470877ed01745831502'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.682799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 891/1779] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:06.683436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ca56a-9308-4e14-948d-c69c1bf7ebf8 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:06.683927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-fa4c161a-af55-4d25-9bd2-98f80d9306f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd384543810ff4470877ed01745831502'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.684613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-fa4c161a-af55-4d25-9bd2-98f80d9306f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd384543810ff4470877ed01745831502'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.684934 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-cd1ca56a-9308-4e14-948d-c69c1bf7ebf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:06.761859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ca56a-9308-4e14-948d-c69c1bf7ebf8 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:06.766435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d25e2-3a1b-4ab3-b547-907678bc8e27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:06.767132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d25e2-3a1b-4ab3-b547-907678bc8e27 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:06.767508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d25e2-3a1b-4ab3-b547-907678bc8e27 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:06.767879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d25e2-3a1b-4ab3-b547-907678bc8e27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:06.770007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-2e9d25e2-3a1b-4ab3-b547-907678bc8e27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:06.786357 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d25e2-3a1b-4ab3-b547-907678bc8e27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:06.787963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d25e2-3a1b-4ab3-b547-907678bc8e27 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:06.792699 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-fa4c161a-af55-4d25-9bd2-98f80d9306f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd384543810ff4470877ed01745831502'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.806362 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 892/1780] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:06.807739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d25e2-3a1b-4ab3-b547-907678bc8e27 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:06.809763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-2e9d25e2-3a1b-4ab3-b547-907678bc8e27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:06.815899 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-fa4c161a-af55-4d25-9bd2-98f80d9306f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd384543810ff4470877ed01745831502'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:06.823368 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d25e2-3a1b-4ab3-b547-907678bc8e27 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:06.833621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ae698-2723-433c-ba2c-3130774bb4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:06.834870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ae698-2723-433c-ba2c-3130774bb4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:06.838375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ae698-2723-433c-ba2c-3130774bb4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:06.838375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ae698-2723-433c-ba2c-3130774bb4e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:06.838375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ae698-2723-433c-ba2c-3130774bb4e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:05:06.838375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ae698-2723-433c-ba2c-3130774bb4e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:06.840933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ae698-2723-433c-ba2c-3130774bb4e0 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:06.841298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 889/1781] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:06 2022] DELETE /identity/v3/users/d384543810ff4470877ed01745831502 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:06.842885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d511bc-959e-402f-942d-c7e7fd75ea3a None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:06.844585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d511bc-959e-402f-942d-c7e7fd75ea3a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:06.844585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d511bc-959e-402f-942d-c7e7fd75ea3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:06.869884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:06.871027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 893/1782] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:05:06.872599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-20d511bc-959e-402f-942d-c7e7fd75ea3a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:06.875950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-20d511bc-959e-402f-942d-c7e7fd75ea3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:06.887609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ae698-2723-433c-ba2c-3130774bb4e0 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:06.907328 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ae698-2723-433c-ba2c-3130774bb4e0 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:06.996830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 890/1783] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:07.005980 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d511bc-959e-402f-942d-c7e7fd75ea3a None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:07.013618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-20d511bc-959e-402f-942d-c7e7fd75ea3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:07.035497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa117b6-7e1b-4177-a92f-8af22ceed08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:07.036454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa117b6-7e1b-4177-a92f-8af22ceed08d admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:07.036955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa117b6-7e1b-4177-a92f-8af22ceed08d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:07.037850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa117b6-7e1b-4177-a92f-8af22ceed08d admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:07.038764 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa117b6-7e1b-4177-a92f-8af22ceed08d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:07.044083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa117b6-7e1b-4177-a92f-8af22ceed08d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:07.051237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:07.161259 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d511bc-959e-402f-942d-c7e7fd75ea3a None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:07.179144 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2367f-e353-4130-8113-cd6e21d42f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:07.180836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2367f-e353-4130-8113-cd6e21d42f06 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:07.181614 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2367f-e353-4130-8113-cd6e21d42f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:07.182542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2367f-e353-4130-8113-cd6e21d42f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:07.215361 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2367f-e353-4130-8113-cd6e21d42f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:07.218448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 894/1784] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:06 2022] POST /identity/v3/projects => generated 381 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:07.221453 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2367f-e353-4130-8113-cd6e21d42f06 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:07.225080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa117b6-7e1b-4177-a92f-8af22ceed08d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:07.263859 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 891/1785] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:07.425536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa117b6-7e1b-4177-a92f-8af22ceed08d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:07.432428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4770606-ccd5-4ea8-af0e-297f0947e544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:07.433330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4770606-ccd5-4ea8-af0e-297f0947e544 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:07.433818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4770606-ccd5-4ea8-af0e-297f0947e544 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:07.435168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4770606-ccd5-4ea8-af0e-297f0947e544 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:07.436493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4770606-ccd5-4ea8-af0e-297f0947e544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:07.440123 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4770606-ccd5-4ea8-af0e-297f0947e544 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:07.549345 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 895/1786] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:07 2022] POST /identity/v3/users => generated 291 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:07.553995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2367f-e353-4130-8113-cd6e21d42f06 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:07.571987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2367f-e353-4130-8113-cd6e21d42f06 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:07.580784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da811b-bcc3-463c-ab63-3e35ff6d7554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:07.582364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da811b-bcc3-463c-ab63-3e35ff6d7554 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:07.582975 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da811b-bcc3-463c-ab63-3e35ff6d7554 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:07.583456 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da811b-bcc3-463c-ab63-3e35ff6d7554 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:07.584844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da811b-bcc3-463c-ab63-3e35ff6d7554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:07.588275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da811b-bcc3-463c-ab63-3e35ff6d7554 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:07.616150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 892/1787] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:07.622541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4770606-ccd5-4ea8-af0e-297f0947e544 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:07.640665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4770606-ccd5-4ea8-af0e-297f0947e544 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:07.649276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e13d8-2685-48c1-8ce8-ebab87a14643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:07.650053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e13d8-2685-48c1-8ce8-ebab87a14643 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:07.651204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e13d8-2685-48c1-8ce8-ebab87a14643 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:07.651204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e13d8-2685-48c1-8ce8-ebab87a14643 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:07.653020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e13d8-2685-48c1-8ce8-ebab87a14643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:07.653915 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e13d8-2685-48c1-8ce8-ebab87a14643 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:07.663435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da811b-bcc3-463c-ab63-3e35ff6d7554 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:07.668856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-19da811b-bcc3-463c-ab63-3e35ff6d7554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:07.674782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 896/1788] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:07.682619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e13d8-2685-48c1-8ce8-ebab87a14643 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:07.704776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da811b-bcc3-463c-ab63-3e35ff6d7554 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:07.713636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e13d8-2685-48c1-8ce8-ebab87a14643 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:07.726707 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:07.727779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:07.727941 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:07.728210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] PATH_INFO: `/v3/projects/eeab432513684a33911fe88335c88984` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:07.735911 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeab432513684a33911fe88335c88984)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:07.738709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:07.744563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30478f-3358-486e-accd-0eba372fae0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:07.745912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30478f-3358-486e-accd-0eba372fae0f admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:07.746250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30478f-3358-486e-accd-0eba372fae0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:07.746613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30478f-3358-486e-accd-0eba372fae0f admin admin] PATH_INFO: `/v3/projects/6058f31a94b64d35909b1ff87cdc98a1/users/f654d7507d0844ffa6e283ecbab93447/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:07.788561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30478f-3358-486e-accd-0eba372fae0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6058f31a94b64d35909b1ff87cdc98a1, user_id=f654d7507d0844ffa6e283ecbab93447, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:07.792816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30478f-3358-486e-accd-0eba372fae0f admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:07.916378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 897/1789] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:07 2022] PUT /identity/v3/projects/6058f31a94b64d35909b1ff87cdc98a1/users/f654d7507d0844ffa6e283ecbab93447/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:07.928582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a300033-3a23-41be-85d0-53e22fab3f34 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:07.929348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a300033-3a23-41be-85d0-53e22fab3f34 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:07.929882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a300033-3a23-41be-85d0-53e22fab3f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:07.938019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeab432513684a33911fe88335c88984', 'request_id': 'req-811cd5d3-4f76-4810-b121-90ebe2e493af'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:07.979418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-4a300033-3a23-41be-85d0-53e22fab3f34 None None] MFA Rules not processed for user `f654d7507d0844ffa6e283ecbab93447`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:07.981312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-4a300033-3a23-41be-85d0-53e22fab3f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:08.056236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeab432513684a33911fe88335c88984', 'request_id': 'req-811cd5d3-4f76-4810-b121-90ebe2e493af'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:08.073437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeab432513684a33911fe88335c88984', 'request_id': 'req-811cd5d3-4f76-4810-b121-90ebe2e493af'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:08.091827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 898/1790] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:08.091827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a300033-3a23-41be-85d0-53e22fab3f34 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:08.092604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-4a300033-3a23-41be-85d0-53e22fab3f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:08.096104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] The token cache is being invalidate because project eeab432513684a33911fe88335c88984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:08.096343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:08.097099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:08.099173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 893/1791] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:07 2022] DELETE /identity/v3/projects/eeab432513684a33911fe88335c88984 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:08.099766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:08.101482 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:08.216331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cd5d3-4f76-4810-b121-90ebe2e493af admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:08.223548 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb2def-717b-4476-83dc-ad3fb0d9d04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:08.224683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb2def-717b-4476-83dc-ad3fb0d9d04b admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:08.225146 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb2def-717b-4476-83dc-ad3fb0d9d04b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:08.225521 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb2def-717b-4476-83dc-ad3fb0d9d04b admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:08.226876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb2def-717b-4476-83dc-ad3fb0d9d04b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:08.230274 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb2def-717b-4476-83dc-ad3fb0d9d04b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:08.236959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a300033-3a23-41be-85d0-53e22fab3f34 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:08.242911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:08.244105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:08.244503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:08.244882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] PATH_INFO: `/v3/projects/57665a3a023246b383edb8d9255610c2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:08.246961 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57665a3a023246b383edb8d9255610c2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:08.250057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:08.382724 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 894/1792] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:08 2022] POST /identity/v3/projects => generated 381 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:08.384689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb2def-717b-4476-83dc-ad3fb0d9d04b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:08.388201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f4cb2def-717b-4476-83dc-ad3fb0d9d04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:08.410774 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57665a3a023246b383edb8d9255610c2', 'request_id': 'req-4d70afcb-6107-4077-a655-193c245c0147'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:08.430689 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57665a3a023246b383edb8d9255610c2', 'request_id': 'req-4d70afcb-6107-4077-a655-193c245c0147'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:08.445201 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57665a3a023246b383edb8d9255610c2', 'request_id': 'req-4d70afcb-6107-4077-a655-193c245c0147'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:08.479350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:08.504002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb2def-717b-4476-83dc-ad3fb0d9d04b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:08.546012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] The token cache is being invalidate because project 57665a3a023246b383edb8d9255610c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:08.546406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:08.547198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:08.553418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 899/1793] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:08 2022] DELETE /identity/v3/projects/57665a3a023246b383edb8d9255610c2 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:08.553418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:08.554575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:08.556346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:08.624157 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:08.625388 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:08.625734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:08.626084 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] PATH_INFO: `/v3/users/bc5b75f0f6724b4e94f646b2eaeb46fa` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:08.627733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc5b75f0f6724b4e94f646b2eaeb46fa)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:08.630464 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:08.660809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70afcb-6107-4077-a655-193c245c0147 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:08.665950 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d913be-50a9-4fda-831f-6f1fbac8aff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:08.666560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d913be-50a9-4fda-831f-6f1fbac8aff7 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:08.667026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d913be-50a9-4fda-831f-6f1fbac8aff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:08.667334 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d913be-50a9-4fda-831f-6f1fbac8aff7 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:08.668025 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d913be-50a9-4fda-831f-6f1fbac8aff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:08.670175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d913be-50a9-4fda-831f-6f1fbac8aff7 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:08.808967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 900/1794] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:08 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:08.810642 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d913be-50a9-4fda-831f-6f1fbac8aff7 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:08.814178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-84d913be-50a9-4fda-831f-6f1fbac8aff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:08.835399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d913be-50a9-4fda-831f-6f1fbac8aff7 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:08.842270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:08.843491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:08.843794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:08.844087 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] PATH_INFO: `/v3/users/afbf40a25f464bbda21afb71ce247044` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:08.845567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] RBAC: Authorizing `identity:delete_user(user_id=afbf40a25f464bbda21afb71ce247044)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:08.848771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:08.902062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc5b75f0f6724b4e94f646b2eaeb46fa'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:08.903450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc5b75f0f6724b4e94f646b2eaeb46fa'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:08.904836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc5b75f0f6724b4e94f646b2eaeb46fa'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:09.039630 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc5b75f0f6724b4e94f646b2eaeb46fa'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:09.060407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc5b75f0f6724b4e94f646b2eaeb46fa'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:09.079681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 895/1795] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:08 2022] DELETE /identity/v3/users/bc5b75f0f6724b4e94f646b2eaeb46fa => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:09.080849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:09.083556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:09.165397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afbf40a25f464bbda21afb71ce247044'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:09.166878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afbf40a25f464bbda21afb71ce247044'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:09.178290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21953f-6184-4ea8-8b02-b88cadee6509 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:09.182707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afbf40a25f464bbda21afb71ce247044'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:09.186544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381c4b4-8220-4498-83c7-577793f00890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:09.191757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381c4b4-8220-4498-83c7-577793f00890 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:09.191757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381c4b4-8220-4498-83c7-577793f00890 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:09.191757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381c4b4-8220-4498-83c7-577793f00890 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:09.191757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381c4b4-8220-4498-83c7-577793f00890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:09.193474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381c4b4-8220-4498-83c7-577793f00890 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:09.205161 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afbf40a25f464bbda21afb71ce247044'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:09.214935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 896/1796] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:09.221129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381c4b4-8220-4498-83c7-577793f00890 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:09.233257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381c4b4-8220-4498-83c7-577793f00890 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:09.239148 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50920eb9-f7d4-4f63-a31c-e42263b85f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:09.240609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50920eb9-f7d4-4f63-a31c-e42263b85f6b admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:09.240609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50920eb9-f7d4-4f63-a31c-e42263b85f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:09.240609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50920eb9-f7d4-4f63-a31c-e42263b85f6b admin admin] PATH_INFO: `/v3/projects/3eb23982aa2243d3afb8827bd9150e4a/users/da8ec84dba364990b3b43f97940e16e3/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:09.258824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50920eb9-f7d4-4f63-a31c-e42263b85f6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb23982aa2243d3afb8827bd9150e4a, user_id=da8ec84dba364990b3b43f97940e16e3, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:09.261987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50920eb9-f7d4-4f63-a31c-e42263b85f6b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:09.262877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afbf40a25f464bbda21afb71ce247044'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:09.272470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 901/1797] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:08 2022] DELETE /identity/v3/users/afbf40a25f464bbda21afb71ce247044 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:09.381694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 897/1798] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:09 2022] PUT /identity/v3/projects/3eb23982aa2243d3afb8827bd9150e4a/users/da8ec84dba364990b3b43f97940e16e3/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:09.387303 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:09.390740 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:09.531678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470caf35-2aaf-4570-9fd9-d0bc2178787d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:09.616230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37ddce-6e3a-4ec7-99b0-3de280144100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:09.617106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37ddce-6e3a-4ec7-99b0-3de280144100 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:09.617402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37ddce-6e3a-4ec7-99b0-3de280144100 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:09.617691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37ddce-6e3a-4ec7-99b0-3de280144100 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:09.618724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37ddce-6e3a-4ec7-99b0-3de280144100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:09.620359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37ddce-6e3a-4ec7-99b0-3de280144100 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:09.636416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 902/1799] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:09.638382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50920eb9-f7d4-4f63-a31c-e42263b85f6b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:09.657818 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50920eb9-f7d4-4f63-a31c-e42263b85f6b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:09.671608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37ddce-6e3a-4ec7-99b0-3de280144100 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:09.681181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ef02b-1fb6-448d-a025-adafae49e8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:09.681181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ef02b-1fb6-448d-a025-adafae49e8fd admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:09.681473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fb37ddce-6e3a-4ec7-99b0-3de280144100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:09.681773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ef02b-1fb6-448d-a025-adafae49e8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:09.681773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ef02b-1fb6-448d-a025-adafae49e8fd admin admin] PATH_INFO: `/v3/projects/3eb23982aa2243d3afb8827bd9150e4a/users/da8ec84dba364990b3b43f97940e16e3/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:09.686275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ef02b-1fb6-448d-a025-adafae49e8fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb23982aa2243d3afb8827bd9150e4a, user_id=da8ec84dba364990b3b43f97940e16e3, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:09.691022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ef02b-1fb6-448d-a025-adafae49e8fd admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:09.708220 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37ddce-6e3a-4ec7-99b0-3de280144100 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:09.717360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 898/1800] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:09 2022] PUT /identity/v3/projects/3eb23982aa2243d3afb8827bd9150e4a/users/da8ec84dba364990b3b43f97940e16e3/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:09.734426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59dca7f-7e90-41ee-ba26-ab863c9fd5ac None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:09.735173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59dca7f-7e90-41ee-ba26-ab863c9fd5ac None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:09.735689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59dca7f-7e90-41ee-ba26-ab863c9fd5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:09.743393 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:09.780455 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e59dca7f-7e90-41ee-ba26-ab863c9fd5ac None None] MFA Rules not processed for user `da8ec84dba364990b3b43f97940e16e3`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:09.782492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e59dca7f-7e90-41ee-ba26-ab863c9fd5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:09.829523 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:09.868605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:09.870025 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:09.871349 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:09.871349 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] PATH_INFO: `/v3/projects/2330c81ef5f646b58767c187eb16b660` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:09.872393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2330c81ef5f646b58767c187eb16b660)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:09.875352 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:09.930590 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 899/1801] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:09.943874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59dca7f-7e90-41ee-ba26-ab863c9fd5ac None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:09.946065 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e59dca7f-7e90-41ee-ba26-ab863c9fd5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:10.072536 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59dca7f-7e90-41ee-ba26-ab863c9fd5ac None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:10.075556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2330c81ef5f646b58767c187eb16b660', 'request_id': 'req-83170b5c-1192-4e26-8bf1-0888fc227d67'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:10.091911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2330c81ef5f646b58767c187eb16b660', 'request_id': 'req-83170b5c-1192-4e26-8bf1-0888fc227d67'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:10.103458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2330c81ef5f646b58767c187eb16b660', 'request_id': 'req-83170b5c-1192-4e26-8bf1-0888fc227d67'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:10.198674 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110a7b6-abab-4f3d-bead-4e4c3c428334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:10.200651 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110a7b6-abab-4f3d-bead-4e4c3c428334 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:10.209885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] The token cache is being invalidate because project 2330c81ef5f646b58767c187eb16b660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:10.210449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110a7b6-abab-4f3d-bead-4e4c3c428334 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:10.210449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110a7b6-abab-4f3d-bead-4e4c3c428334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:10.211479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:10.212934 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:10.215509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 903/1802] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:09 2022] DELETE /identity/v3/projects/2330c81ef5f646b58767c187eb16b660 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:10.216666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:10.219000 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:10.352651 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110a7b6-abab-4f3d-bead-4e4c3c428334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:10.354886 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110a7b6-abab-4f3d-bead-4e4c3c428334 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:10.372051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83170b5c-1192-4e26-8bf1-0888fc227d67 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:10.374112 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 900/1803] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:05:10.376979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3482db-14ac-43e9-bca3-64bf1c60c484 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:10.377623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3482db-14ac-43e9-bca3-64bf1c60c484 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:10.378030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3482db-14ac-43e9-bca3-64bf1c60c484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:10.383892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203970fa-2bbd-4dcc-949b-3cec30cf5b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:10.385249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203970fa-2bbd-4dcc-949b-3cec30cf5b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:10.385558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203970fa-2bbd-4dcc-949b-3cec30cf5b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:10.385933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203970fa-2bbd-4dcc-949b-3cec30cf5b09 admin admin] PATH_INFO: `/v3/projects/721c63a11fe3470eb30eef8363826b5c` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:10.387810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203970fa-2bbd-4dcc-949b-3cec30cf5b09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=721c63a11fe3470eb30eef8363826b5c)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:10.390861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203970fa-2bbd-4dcc-949b-3cec30cf5b09 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:10.423535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3d3482db-14ac-43e9-bca3-64bf1c60c484 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:10.425650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3d3482db-14ac-43e9-bca3-64bf1c60c484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:10.440381 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 901/1804] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:10.449628 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4b842-d0be-40b0-978e-d553c40268ce None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:10.449945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4b842-d0be-40b0-978e-d553c40268ce None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:10.450224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4b842-d0be-40b0-978e-d553c40268ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:10.472867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-bda4b842-d0be-40b0-978e-d553c40268ce None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:10.474105 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bda4b842-d0be-40b0-978e-d553c40268ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:10.492346 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 902/1805] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:10.498826 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4b842-d0be-40b0-978e-d553c40268ce None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:10.508556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4b842-d0be-40b0-978e-d553c40268ce None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:10.514833 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26740999-db0d-4077-b412-16e95861a4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:10.515582 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26740999-db0d-4077-b412-16e95861a4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:10.515900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26740999-db0d-4077-b412-16e95861a4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:10.516405 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26740999-db0d-4077-b412-16e95861a4f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:10.517369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26740999-db0d-4077-b412-16e95861a4f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:05:10.517704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26740999-db0d-4077-b412-16e95861a4f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:10.519391 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26740999-db0d-4077-b412-16e95861a4f2 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:10.537569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 903/1806] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:05:10.544987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26740999-db0d-4077-b412-16e95861a4f2 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:10.557819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26740999-db0d-4077-b412-16e95861a4f2 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:10.568121 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b78fe-5ade-4137-b130-417330aa2d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:10.569052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b78fe-5ade-4137-b130-417330aa2d58 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:10.569707 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b78fe-5ade-4137-b130-417330aa2d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:10.569819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b78fe-5ade-4137-b130-417330aa2d58 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:10.570575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b78fe-5ade-4137-b130-417330aa2d58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:10.572898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b78fe-5ade-4137-b130-417330aa2d58 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:10.610833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-203970fa-2bbd-4dcc-949b-3cec30cf5b09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '721c63a11fe3470eb30eef8363826b5c', 'request_id': 'req-203970fa-2bbd-4dcc-949b-3cec30cf5b09'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:10.619426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-203970fa-2bbd-4dcc-949b-3cec30cf5b09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '721c63a11fe3470eb30eef8363826b5c', 'request_id': 'req-203970fa-2bbd-4dcc-949b-3cec30cf5b09'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:10.627325 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-203970fa-2bbd-4dcc-949b-3cec30cf5b09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '721c63a11fe3470eb30eef8363826b5c', 'request_id': 'req-203970fa-2bbd-4dcc-949b-3cec30cf5b09'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:10.634043 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:10.679234 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 904/1807] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:10 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:05:10.684108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b78fe-5ade-4137-b130-417330aa2d58 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:10.776214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b78fe-5ade-4137-b130-417330aa2d58 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:10.785486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9798d-1456-445f-b640-f997feb9f290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:10.786511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9798d-1456-445f-b640-f997feb9f290 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:10.786813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9798d-1456-445f-b640-f997feb9f290 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:10.787316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9798d-1456-445f-b640-f997feb9f290 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:10.787888 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9798d-1456-445f-b640-f997feb9f290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:10.790481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9798d-1456-445f-b640-f997feb9f290 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:10.795585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-203970fa-2bbd-4dcc-949b-3cec30cf5b09 admin admin] The token cache is being invalidate because project 721c63a11fe3470eb30eef8363826b5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:10.795938 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-203970fa-2bbd-4dcc-949b-3cec30cf5b09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:10.800132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-203970fa-2bbd-4dcc-949b-3cec30cf5b09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:10.809044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 904/1808] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:10 2022] DELETE /identity/v3/projects/721c63a11fe3470eb30eef8363826b5c => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:10.817149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a7061-5424-404a-abb4-a888e6765865 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:10.817597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a7061-5424-404a-abb4-a888e6765865 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:10.824079 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a7061-5424-404a-abb4-a888e6765865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:10.847593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-3a7a7061-5424-404a-abb4-a888e6765865 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:10.849552 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3a7a7061-5424-404a-abb4-a888e6765865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:10.851621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:10.865697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 905/1809] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:10.877416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b86c2-643d-4ddc-bb92-0d050d8adb33 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:10.877776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b86c2-643d-4ddc-bb92-0d050d8adb33 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:10.878425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b86c2-643d-4ddc-bb92-0d050d8adb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:10.891731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 905/1810] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:10 2022] POST /identity/v3/users => generated 294 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:10.898640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9798d-1456-445f-b640-f997feb9f290 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:10.904301 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ece9798d-1456-445f-b640-f997feb9f290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:10.911279 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-1d6b86c2-643d-4ddc-bb92-0d050d8adb33 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:10.913104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1d6b86c2-643d-4ddc-bb92-0d050d8adb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:10.929210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 906/1811] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:10.935186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9798d-1456-445f-b640-f997feb9f290 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:10.936314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b86c2-643d-4ddc-bb92-0d050d8adb33 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:10.941221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b770ea9-058f-42e0-b3dc-e3401ebb3ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:10.941749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b770ea9-058f-42e0-b3dc-e3401ebb3ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:10.942104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b770ea9-058f-42e0-b3dc-e3401ebb3ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:10.942395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b770ea9-058f-42e0-b3dc-e3401ebb3ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:10.943314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b770ea9-058f-42e0-b3dc-e3401ebb3ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:10.945879 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b770ea9-058f-42e0-b3dc-e3401ebb3ef2 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:10.950055 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b86c2-643d-4ddc-bb92-0d050d8adb33 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:10.955889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645c89d-b367-40b7-b46d-b11fcba18a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:10.956673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645c89d-b367-40b7-b46d-b11fcba18a17 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:10.956969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645c89d-b367-40b7-b46d-b11fcba18a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:10.971080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645c89d-b367-40b7-b46d-b11fcba18a17 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:10.971080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645c89d-b367-40b7-b46d-b11fcba18a17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:05:10.971613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 906/1812] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:10.971728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645c89d-b367-40b7-b46d-b11fcba18a17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:10.971728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645c89d-b367-40b7-b46d-b11fcba18a17 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:10.974393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 907/1813] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:05:10.978945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b770ea9-058f-42e0-b3dc-e3401ebb3ef2 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:10.980486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645c89d-b367-40b7-b46d-b11fcba18a17 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:10.991337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b770ea9-058f-42e0-b3dc-e3401ebb3ef2 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:10.993608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645c89d-b367-40b7-b46d-b11fcba18a17 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:10.998031 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe4afe-682f-4a80-822b-c413b0abcc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:10.998821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe4afe-682f-4a80-822b-c413b0abcc17 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:10.999074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe4afe-682f-4a80-822b-c413b0abcc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:10.999416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe4afe-682f-4a80-822b-c413b0abcc17 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:11.000717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe4afe-682f-4a80-822b-c413b0abcc17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:11.000880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a41cfa-736e-4a0e-984b-5d2e8d4d08af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:11.001819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a41cfa-736e-4a0e-984b-5d2e8d4d08af admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:11.002218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a41cfa-736e-4a0e-984b-5d2e8d4d08af admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:11.002676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe4afe-682f-4a80-822b-c413b0abcc17 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:11.002811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a41cfa-736e-4a0e-984b-5d2e8d4d08af admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:11.003653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a41cfa-736e-4a0e-984b-5d2e8d4d08af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:11.006823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a41cfa-736e-4a0e-984b-5d2e8d4d08af admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:11.020803 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 907/1814] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:11.025372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe4afe-682f-4a80-822b-c413b0abcc17 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:11.042548 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe4afe-682f-4a80-822b-c413b0abcc17 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:11.051390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65076480-bc16-4c45-9917-f0a586c1f6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:11.052310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65076480-bc16-4c45-9917-f0a586c1f6c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:11.052599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65076480-bc16-4c45-9917-f0a586c1f6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:11.052883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65076480-bc16-4c45-9917-f0a586c1f6c8 admin admin] PATH_INFO: `/v3/projects/5d8b1ca2b776409eb37c94a44cc25dd4/users/e84882a049f94b6dbcd644f6c1210d12/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:11.073091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65076480-bc16-4c45-9917-f0a586c1f6c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d8b1ca2b776409eb37c94a44cc25dd4, user_id=e84882a049f94b6dbcd644f6c1210d12, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:11.076753 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65076480-bc16-4c45-9917-f0a586c1f6c8 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:11.144696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 908/1815] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:10 2022] POST /identity/v3/projects => generated 365 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:11.149927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a41cfa-736e-4a0e-984b-5d2e8d4d08af admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:11.224178 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 908/1816] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:11 2022] PUT /identity/v3/projects/5d8b1ca2b776409eb37c94a44cc25dd4/users/e84882a049f94b6dbcd644f6c1210d12/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:11.224178 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65076480-bc16-4c45-9917-f0a586c1f6c8 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:11.224178 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-65076480-bc16-4c45-9917-f0a586c1f6c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:11.346534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a41cfa-736e-4a0e-984b-5d2e8d4d08af admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:11.390575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65076480-bc16-4c45-9917-f0a586c1f6c8 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:11.399095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:11.400356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:11.400874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:11.401383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] PATH_INFO: `/v3/users/f654d7507d0844ffa6e283ecbab93447` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:11.407380 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f654d7507d0844ffa6e283ecbab93447)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:11.413947 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:11.496769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9842b0a-e987-447e-bb4a-cc56ce18393a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:11.498580 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9842b0a-e987-447e-bb4a-cc56ce18393a admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:11.499456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9842b0a-e987-447e-bb4a-cc56ce18393a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:11.500697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9842b0a-e987-447e-bb4a-cc56ce18393a admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:11.503342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9842b0a-e987-447e-bb4a-cc56ce18393a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:11.506012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9842b0a-e987-447e-bb4a-cc56ce18393a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:11.634638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 909/1817] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:11 2022] POST /identity/v3/users => generated 283 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:05:11.640496 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611a788-8364-406b-8722-70fecc232876 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:11.641099 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611a788-8364-406b-8722-70fecc232876 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:11.641597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611a788-8364-406b-8722-70fecc232876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:11.693228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f654d7507d0844ffa6e283ecbab93447'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:11.693978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f654d7507d0844ffa6e283ecbab93447'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:11.694642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f654d7507d0844ffa6e283ecbab93447'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:11.698358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-3611a788-8364-406b-8722-70fecc232876 None None] MFA Rules not processed for user `e84882a049f94b6dbcd644f6c1210d12`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:11.700193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3611a788-8364-406b-8722-70fecc232876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:11.710722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=5 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:11.785128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f654d7507d0844ffa6e283ecbab93447'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:11.797957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 910/1818] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:11.798886 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611a788-8364-406b-8722-70fecc232876 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:11.800019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f654d7507d0844ffa6e283ecbab93447'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:11.803020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3611a788-8364-406b-8722-70fecc232876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:11.815874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 909/1819] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:11 2022] DELETE /identity/v3/users/f654d7507d0844ffa6e283ecbab93447 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:11.817030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:11.820023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:11.823697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611a788-8364-406b-8722-70fecc232876 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:11.900085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531945c4-8ea0-49f9-b784-4cfd19c8d69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:11.901076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531945c4-8ea0-49f9-b784-4cfd19c8d69b admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:11.901468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531945c4-8ea0-49f9-b784-4cfd19c8d69b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:11.901883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531945c4-8ea0-49f9-b784-4cfd19c8d69b admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:11.902893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531945c4-8ea0-49f9-b784-4cfd19c8d69b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:11.904882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531945c4-8ea0-49f9-b784-4cfd19c8d69b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:11.913553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191bcdd-f775-4476-b03c-fcef2dc1c2aa admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:11.920651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 911/1820] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:11.924649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91933980-efc5-4ebc-9b03-3bd8941a0864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:11.924649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91933980-efc5-4ebc-9b03-3bd8941a0864 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:11.924649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91933980-efc5-4ebc-9b03-3bd8941a0864 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:11.924649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91933980-efc5-4ebc-9b03-3bd8941a0864 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:11.924958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37a316-4a94-4daa-a1b4-4adc27097218 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:11.925050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91933980-efc5-4ebc-9b03-3bd8941a0864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:11.925120 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37a316-4a94-4daa-a1b4-4adc27097218 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:11.925120 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37a316-4a94-4daa-a1b4-4adc27097218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:11.926365 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91933980-efc5-4ebc-9b03-3bd8941a0864 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:11.952870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-0c37a316-4a94-4daa-a1b4-4adc27097218 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:11.954527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-0c37a316-4a94-4daa-a1b4-4adc27097218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:11.962759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 912/1821] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:11.964094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37a316-4a94-4daa-a1b4-4adc27097218 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:11.970051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:11.976576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37a316-4a94-4daa-a1b4-4adc27097218 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:11.982408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d65f14-b9b5-40df-88b7-2bfdacb1198b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:11.983427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d65f14-b9b5-40df-88b7-2bfdacb1198b admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:11.984493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d65f14-b9b5-40df-88b7-2bfdacb1198b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:11.984943 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d65f14-b9b5-40df-88b7-2bfdacb1198b admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:11.986192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d65f14-b9b5-40df-88b7-2bfdacb1198b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:11.988062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d65f14-b9b5-40df-88b7-2bfdacb1198b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:11.999769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 913/1822] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:12.000703 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d65f14-b9b5-40df-88b7-2bfdacb1198b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:12.002449 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b3d65f14-b9b5-40df-88b7-2bfdacb1198b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:12.073814 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 910/1823] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:11 2022] POST /identity/v3/projects => generated 387 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:12.074798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91933980-efc5-4ebc-9b03-3bd8941a0864 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:12.080826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d65f14-b9b5-40df-88b7-2bfdacb1198b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:12.141010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68462f42-6895-4d13-b8da-680194898d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:12.141588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68462f42-6895-4d13-b8da-680194898d66 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:12.141876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68462f42-6895-4d13-b8da-680194898d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:12.142157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68462f42-6895-4d13-b8da-680194898d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:12.152253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91933980-efc5-4ebc-9b03-3bd8941a0864 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:12.152842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68462f42-6895-4d13-b8da-680194898d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:12.159271 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68462f42-6895-4d13-b8da-680194898d66 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:12.164376 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de22ee-a47b-4554-a964-3fdab4c39506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:12.164786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de22ee-a47b-4554-a964-3fdab4c39506 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:12.165198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de22ee-a47b-4554-a964-3fdab4c39506 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:12.165486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de22ee-a47b-4554-a964-3fdab4c39506 admin admin] PATH_INFO: `/v3/projects/dc9aa94d422144f786b359a94c0154e6/users/a5e50c563c9741c3a06b474f8636eb99/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:12.167416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 914/1824] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:12.167832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68462f42-6895-4d13-b8da-680194898d66 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:12.180383 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68462f42-6895-4d13-b8da-680194898d66 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:12.187199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de22ee-a47b-4554-a964-3fdab4c39506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9aa94d422144f786b359a94c0154e6, user_id=a5e50c563c9741c3a06b474f8636eb99, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:12.190803 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34e970-2a6c-413c-9232-8f9e214e821e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:12.191217 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de22ee-a47b-4554-a964-3fdab4c39506 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:12.191823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34e970-2a6c-413c-9232-8f9e214e821e admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:12.192185 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34e970-2a6c-413c-9232-8f9e214e821e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:12.192777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34e970-2a6c-413c-9232-8f9e214e821e admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:12.194484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34e970-2a6c-413c-9232-8f9e214e821e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:12.196770 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34e970-2a6c-413c-9232-8f9e214e821e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:12.298092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 911/1825] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:12 2022] PUT /identity/v3/projects/dc9aa94d422144f786b359a94c0154e6/users/a5e50c563c9741c3a06b474f8636eb99/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:12.308694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17507104-3870-4ac6-a7fb-785b2804c62d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:12.308981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17507104-3870-4ac6-a7fb-785b2804c62d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:12.309268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17507104-3870-4ac6-a7fb-785b2804c62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:12.335466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 915/1826] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:12 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:12.346104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34e970-2a6c-413c-9232-8f9e214e821e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:12.354718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-17507104-3870-4ac6-a7fb-785b2804c62d None None] MFA Rules not processed for user `a5e50c563c9741c3a06b474f8636eb99`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:12.357013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-17507104-3870-4ac6-a7fb-785b2804c62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:12.363735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34e970-2a6c-413c-9232-8f9e214e821e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:12.471980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591d809-6d6c-4a0a-98cc-cf40cdd9b170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:12.472949 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591d809-6d6c-4a0a-98cc-cf40cdd9b170 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:12.473469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591d809-6d6c-4a0a-98cc-cf40cdd9b170 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:12.473928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591d809-6d6c-4a0a-98cc-cf40cdd9b170 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:12.475467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591d809-6d6c-4a0a-98cc-cf40cdd9b170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:12.477369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591d809-6d6c-4a0a-98cc-cf40cdd9b170 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:12.497010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 916/1827] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:12.500819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 912/1828] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:12.507124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591d809-6d6c-4a0a-98cc-cf40cdd9b170 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:12.510115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17507104-3870-4ac6-a7fb-785b2804c62d None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:12.518888 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591d809-6d6c-4a0a-98cc-cf40cdd9b170 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:12.525390 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727ecae-f70a-496f-a107-aeb3edd75ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:12.526943 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727ecae-f70a-496f-a107-aeb3edd75ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:12.527382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727ecae-f70a-496f-a107-aeb3edd75ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:12.527989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727ecae-f70a-496f-a107-aeb3edd75ee5 admin admin] PATH_INFO: `/v3/projects/5ba1c78d6bdb4faebe9ce40ec6a3b530/users/51af43676ba2410ea8de4e660ffe597f/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:12.556737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17507104-3870-4ac6-a7fb-785b2804c62d None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:12.566957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dc47a-243d-4dfa-b60c-58fe9ce121d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:12.568133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dc47a-243d-4dfa-b60c-58fe9ce121d6 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:12.568853 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dc47a-243d-4dfa-b60c-58fe9ce121d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:12.569354 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dc47a-243d-4dfa-b60c-58fe9ce121d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:12.569632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727ecae-f70a-496f-a107-aeb3edd75ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba1c78d6bdb4faebe9ce40ec6a3b530, user_id=51af43676ba2410ea8de4e660ffe597f, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:12.571418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dc47a-243d-4dfa-b60c-58fe9ce121d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:12.573161 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727ecae-f70a-496f-a107-aeb3edd75ee5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:12.575040 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dc47a-243d-4dfa-b60c-58fe9ce121d6 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:12.675857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 917/1829] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:12 2022] PUT /identity/v3/projects/5ba1c78d6bdb4faebe9ce40ec6a3b530/users/51af43676ba2410ea8de4e660ffe597f/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:12.680849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727ecae-f70a-496f-a107-aeb3edd75ee5 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:12.710735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 913/1830] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:12 2022] POST /identity/v3/projects => generated 363 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:12.714975 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dc47a-243d-4dfa-b60c-58fe9ce121d6 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:12.825330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727ecae-f70a-496f-a107-aeb3edd75ee5 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:12.879074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dc47a-243d-4dfa-b60c-58fe9ce121d6 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:12.881064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:12.891596 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d223bd6-c7ff-448f-ac34-99c7bae6a643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:12.893527 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d223bd6-c7ff-448f-ac34-99c7bae6a643 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:12.894579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d223bd6-c7ff-448f-ac34-99c7bae6a643 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:12.894932 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d223bd6-c7ff-448f-ac34-99c7bae6a643 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:12.896293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d223bd6-c7ff-448f-ac34-99c7bae6a643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:12.898741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d223bd6-c7ff-448f-ac34-99c7bae6a643 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:12.936433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492ef86-01d2-4bb4-9c73-fd69959b69f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:12.937213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492ef86-01d2-4bb4-9c73-fd69959b69f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:12.937693 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492ef86-01d2-4bb4-9c73-fd69959b69f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:12.938199 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492ef86-01d2-4bb4-9c73-fd69959b69f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:12.939267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492ef86-01d2-4bb4-9c73-fd69959b69f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:12.941287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492ef86-01d2-4bb4-9c73-fd69959b69f3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:12.959846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 918/1831] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:12.960885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492ef86-01d2-4bb4-9c73-fd69959b69f3 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:12.962849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5492ef86-01d2-4bb4-9c73-fd69959b69f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:12.982026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492ef86-01d2-4bb4-9c73-fd69959b69f3 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:12.993757 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:12.995341 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:12.995820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:12.996531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] PATH_INFO: `/v3/projects/6058f31a94b64d35909b1ff87cdc98a1` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:13.000056 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6058f31a94b64d35909b1ff87cdc98a1)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:13.001652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:13.049473 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 914/1832] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:12 2022] POST /identity/v3/users => generated 288 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:05:13.050661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d223bd6-c7ff-448f-ac34-99c7bae6a643 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:13.066907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d223bd6-c7ff-448f-ac34-99c7bae6a643 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:13.073540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1325d9b-b0c6-40ce-b804-c7377a39d456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:13.074454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1325d9b-b0c6-40ce-b804-c7377a39d456 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:13.074823 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1325d9b-b0c6-40ce-b804-c7377a39d456 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:13.075249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1325d9b-b0c6-40ce-b804-c7377a39d456 admin admin] PATH_INFO: `/v3/projects/5ba1c78d6bdb4faebe9ce40ec6a3b530/users/51af43676ba2410ea8de4e660ffe597f/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:13.077226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1325d9b-b0c6-40ce-b804-c7377a39d456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba1c78d6bdb4faebe9ce40ec6a3b530, user_id=51af43676ba2410ea8de4e660ffe597f, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:13.080757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1325d9b-b0c6-40ce-b804-c7377a39d456 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:13.093271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 915/1833] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:13 2022] PUT /identity/v3/projects/5ba1c78d6bdb4faebe9ce40ec6a3b530/users/51af43676ba2410ea8de4e660ffe597f/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:13.094396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1325d9b-b0c6-40ce-b804-c7377a39d456 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:13.167310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1325d9b-b0c6-40ce-b804-c7377a39d456 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:13.169135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6058f31a94b64d35909b1ff87cdc98a1', 'request_id': 'req-f39d90e0-ab11-471e-b8e1-ae3741db9229'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:13.179310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6058f31a94b64d35909b1ff87cdc98a1', 'request_id': 'req-f39d90e0-ab11-471e-b8e1-ae3741db9229'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:13.186287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6058f31a94b64d35909b1ff87cdc98a1', 'request_id': 'req-f39d90e0-ab11-471e-b8e1-ae3741db9229'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:13.239132 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b78d8-b801-4660-8dcc-448c88e1b651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:13.239895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b78d8-b801-4660-8dcc-448c88e1b651 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:13.240193 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b78d8-b801-4660-8dcc-448c88e1b651 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:13.240476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b78d8-b801-4660-8dcc-448c88e1b651 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:13.241195 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b78d8-b801-4660-8dcc-448c88e1b651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:13.243183 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b78d8-b801-4660-8dcc-448c88e1b651 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:13.254942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 916/1834] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:13.256434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de77a7-150e-4889-ad8a-203c463a8084 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:13.256676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de77a7-150e-4889-ad8a-203c463a8084 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:13.256951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de77a7-150e-4889-ad8a-203c463a8084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:13.269094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] The token cache is being invalidate because project 6058f31a94b64d35909b1ff87cdc98a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:13.269769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:13.270745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:13.273476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 919/1835] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:12 2022] DELETE /identity/v3/projects/6058f31a94b64d35909b1ff87cdc98a1 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:13.274608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:13.274748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:13.277061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:13.293080 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b8de77a7-150e-4889-ad8a-203c463a8084 None None] MFA Rules not processed for user `51af43676ba2410ea8de4e660ffe597f`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:13.294363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b8de77a7-150e-4889-ad8a-203c463a8084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:13.296171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d90e0-ab11-471e-b8e1-ae3741db9229 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:13.303513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51216b50-2242-4b19-925f-70332e793914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:13.304441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51216b50-2242-4b19-925f-70332e793914 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:13.304725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51216b50-2242-4b19-925f-70332e793914 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:13.304999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51216b50-2242-4b19-925f-70332e793914 admin admin] PATH_INFO: `/v3/projects/81ff84d4e534408e8e47b1de5a7bdbf8/users/9db0ead52a3b4c4083fc1b06aa2628c4/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:13.328017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51216b50-2242-4b19-925f-70332e793914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ff84d4e534408e8e47b1de5a7bdbf8, user_id=9db0ead52a3b4c4083fc1b06aa2628c4, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:13.330861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51216b50-2242-4b19-925f-70332e793914 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:13.376478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 917/1836] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:13.378754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de77a7-150e-4889-ad8a-203c463a8084 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:13.380998 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b8de77a7-150e-4889-ad8a-203c463a8084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:13.401059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de77a7-150e-4889-ad8a-203c463a8084 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:13.406876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490feef3-73e5-4061-9533-8e5e7f1f7a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:13.408045 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490feef3-73e5-4061-9533-8e5e7f1f7a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:13.408425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490feef3-73e5-4061-9533-8e5e7f1f7a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:13.408731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490feef3-73e5-4061-9533-8e5e7f1f7a11 admin admin] PATH_INFO: `/v3/users/da8ec84dba364990b3b43f97940e16e3` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:13.410243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490feef3-73e5-4061-9533-8e5e7f1f7a11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da8ec84dba364990b3b43f97940e16e3)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:13.412403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490feef3-73e5-4061-9533-8e5e7f1f7a11 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:13.413260 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 920/1837] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:13 2022] PUT /identity/v3/projects/81ff84d4e534408e8e47b1de5a7bdbf8/users/9db0ead52a3b4c4083fc1b06aa2628c4/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:13.418286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51216b50-2242-4b19-925f-70332e793914 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:13.421901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-51216b50-2242-4b19-925f-70332e793914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:13.496606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51216b50-2242-4b19-925f-70332e793914 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:13.501395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6acfb1-59e7-439f-a8cc-9c65aa25cb14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:13.502082 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6acfb1-59e7-439f-a8cc-9c65aa25cb14 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:13.502463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6acfb1-59e7-439f-a8cc-9c65aa25cb14 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:13.502678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6acfb1-59e7-439f-a8cc-9c65aa25cb14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:13.525513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:13.597789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6acfb1-59e7-439f-a8cc-9c65aa25cb14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:13.599626 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6acfb1-59e7-439f-a8cc-9c65aa25cb14 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:13.620490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 921/1838] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:13.621853 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6acfb1-59e7-439f-a8cc-9c65aa25cb14 service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:13.689748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-490feef3-73e5-4061-9533-8e5e7f1f7a11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da8ec84dba364990b3b43f97940e16e3'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:13.691533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-490feef3-73e5-4061-9533-8e5e7f1f7a11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da8ec84dba364990b3b43f97940e16e3'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:13.706018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-490feef3-73e5-4061-9533-8e5e7f1f7a11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da8ec84dba364990b3b43f97940e16e3'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:13.706648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6acfb1-59e7-439f-a8cc-9c65aa25cb14 service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:13.706648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72733c23-a3d6-499a-a1a3-30d516003efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:13.706648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72733c23-a3d6-499a-a1a3-30d516003efd admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:13.706648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72733c23-a3d6-499a-a1a3-30d516003efd admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:13.707150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72733c23-a3d6-499a-a1a3-30d516003efd admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:13.707150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72733c23-a3d6-499a-a1a3-30d516003efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:13.708697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72733c23-a3d6-499a-a1a3-30d516003efd admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:13.722369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 922/1839] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:13.727498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72733c23-a3d6-499a-a1a3-30d516003efd admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:13.730955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-72733c23-a3d6-499a-a1a3-30d516003efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:13.758021 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72733c23-a3d6-499a-a1a3-30d516003efd admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:13.766699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f8559-e17a-4220-bc14-ea027b02f6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:13.770616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f8559-e17a-4220-bc14-ea027b02f6d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:13.770913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f8559-e17a-4220-bc14-ea027b02f6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:13.776271 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f8559-e17a-4220-bc14-ea027b02f6d2 admin admin] PATH_INFO: `/v3/projects/81ff84d4e534408e8e47b1de5a7bdbf8/users/9db0ead52a3b4c4083fc1b06aa2628c4/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:13.776271 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f8559-e17a-4220-bc14-ea027b02f6d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ff84d4e534408e8e47b1de5a7bdbf8, user_id=9db0ead52a3b4c4083fc1b06aa2628c4, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:13.779987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f8559-e17a-4220-bc14-ea027b02f6d2 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:13.797822 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-490feef3-73e5-4061-9533-8e5e7f1f7a11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da8ec84dba364990b3b43f97940e16e3'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:13.817408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 923/1840] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:13 2022] PUT /identity/v3/projects/81ff84d4e534408e8e47b1de5a7bdbf8/users/9db0ead52a3b4c4083fc1b06aa2628c4/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:13.818175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f8559-e17a-4220-bc14-ea027b02f6d2 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:13.820025 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7f9f8559-e17a-4220-bc14-ea027b02f6d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:13.826384 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-490feef3-73e5-4061-9533-8e5e7f1f7a11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da8ec84dba364990b3b43f97940e16e3'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:13.845778 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 918/1841] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:13 2022] DELETE /identity/v3/users/da8ec84dba364990b3b43f97940e16e3 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:13.847657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05fba2-d911-499c-9f86-1c804381d14f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:13.848160 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05fba2-d911-499c-9f86-1c804381d14f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:13.848666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05fba2-d911-499c-9f86-1c804381d14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:13.891817 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-9a05fba2-d911-499c-9f86-1c804381d14f None None] MFA Rules not processed for user `9db0ead52a3b4c4083fc1b06aa2628c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:13.893634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9a05fba2-d911-499c-9f86-1c804381d14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:13.935095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f8559-e17a-4220-bc14-ea027b02f6d2 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:14.002642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 919/1842] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:13 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:14.011975 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:14.021487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05fba2-d911-499c-9f86-1c804381d14f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:14.023753 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9a05fba2-d911-499c-9f86-1c804381d14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:14.036368 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746bc48-72c1-40ed-9f24-ae22f38a249e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:14.037297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746bc48-72c1-40ed-9f24-ae22f38a249e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:14.037644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746bc48-72c1-40ed-9f24-ae22f38a249e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:14.038044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746bc48-72c1-40ed-9f24-ae22f38a249e admin admin] PATH_INFO: `/v3/users/e84882a049f94b6dbcd644f6c1210d12` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:14.039490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746bc48-72c1-40ed-9f24-ae22f38a249e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e84882a049f94b6dbcd644f6c1210d12)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:14.042301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746bc48-72c1-40ed-9f24-ae22f38a249e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:14.136235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05fba2-d911-499c-9f86-1c804381d14f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:14.148481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b65378-a0bd-4f6e-8504-e8ab9bf9d76a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:14.149114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b65378-a0bd-4f6e-8504-e8ab9bf9d76a service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:14.151197 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b65378-a0bd-4f6e-8504-e8ab9bf9d76a service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:14.151488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b65378-a0bd-4f6e-8504-e8ab9bf9d76a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:14.174154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b65378-a0bd-4f6e-8504-e8ab9bf9d76a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:14.175717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b65378-a0bd-4f6e-8504-e8ab9bf9d76a service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:14.190108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 920/1843] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:14 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:14.340673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2746bc48-72c1-40ed-9f24-ae22f38a249e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e84882a049f94b6dbcd644f6c1210d12'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:14.342308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2746bc48-72c1-40ed-9f24-ae22f38a249e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e84882a049f94b6dbcd644f6c1210d12'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:14.357897 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2746bc48-72c1-40ed-9f24-ae22f38a249e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e84882a049f94b6dbcd644f6c1210d12'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:14.386987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2746bc48-72c1-40ed-9f24-ae22f38a249e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e84882a049f94b6dbcd644f6c1210d12'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:14.537415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b65378-a0bd-4f6e-8504-e8ab9bf9d76a service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:14.545725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2746bc48-72c1-40ed-9f24-ae22f38a249e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e84882a049f94b6dbcd644f6c1210d12'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:14.552785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b65378-a0bd-4f6e-8504-e8ab9bf9d76a service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:14.555813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 924/1844] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:13 2022] DELETE /identity/v3/users/e84882a049f94b6dbcd644f6c1210d12 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:14.557042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ae7f6-aabd-4aec-944e-bac25ab5f256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:14.557714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ae7f6-aabd-4aec-944e-bac25ab5f256 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:14.558014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ae7f6-aabd-4aec-944e-bac25ab5f256 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:14.558300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ae7f6-aabd-4aec-944e-bac25ab5f256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:14.561006 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000143f-582c-46e0-a413-f367be001613 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:14.561223 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ea9ae7f6-aabd-4aec-944e-bac25ab5f256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:14.561369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000143f-582c-46e0-a413-f367be001613 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:14.561748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000143f-582c-46e0-a413-f367be001613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:14.583630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-0000143f-582c-46e0-a413-f367be001613 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:14.585263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-0000143f-582c-46e0-a413-f367be001613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:14.661373 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ae7f6-aabd-4aec-944e-bac25ab5f256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:14.662919 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ae7f6-aabd-4aec-944e-bac25ab5f256 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:14.671596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 925/1845] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:14.677090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 921/1846] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:05:14.686112 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000143f-582c-46e0-a413-f367be001613 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:14.688167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-0000143f-582c-46e0-a413-f367be001613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:14.708430 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ae7f6-aabd-4aec-944e-bac25ab5f256 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:14.709978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ea9ae7f6-aabd-4aec-944e-bac25ab5f256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:14.731937 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ae7f6-aabd-4aec-944e-bac25ab5f256 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:14.739201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960abd87-0c62-45b6-8d96-47954d7bf962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:14.739983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960abd87-0c62-45b6-8d96-47954d7bf962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:14.740249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960abd87-0c62-45b6-8d96-47954d7bf962 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:14.740582 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960abd87-0c62-45b6-8d96-47954d7bf962 admin admin] PATH_INFO: `/v3/projects/3eb23982aa2243d3afb8827bd9150e4a` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:14.741730 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960abd87-0c62-45b6-8d96-47954d7bf962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eb23982aa2243d3afb8827bd9150e4a)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:14.743870 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960abd87-0c62-45b6-8d96-47954d7bf962 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:14.779281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:14.791104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000143f-582c-46e0-a413-f367be001613 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:14.796306 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf69f82-e327-4951-b285-ff3b8411db6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:14.797307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf69f82-e327-4951-b285-ff3b8411db6b service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:14.800828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf69f82-e327-4951-b285-ff3b8411db6b service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:14.800828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf69f82-e327-4951-b285-ff3b8411db6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:14.810758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf69f82-e327-4951-b285-ff3b8411db6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:14.812902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf69f82-e327-4951-b285-ff3b8411db6b service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:14.827547 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 926/1847] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:05:14.965780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-960abd87-0c62-45b6-8d96-47954d7bf962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eb23982aa2243d3afb8827bd9150e4a', 'request_id': 'req-960abd87-0c62-45b6-8d96-47954d7bf962'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:15.074216 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-960abd87-0c62-45b6-8d96-47954d7bf962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eb23982aa2243d3afb8827bd9150e4a', 'request_id': 'req-960abd87-0c62-45b6-8d96-47954d7bf962'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:15.090177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-960abd87-0c62-45b6-8d96-47954d7bf962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eb23982aa2243d3afb8827bd9150e4a', 'request_id': 'req-960abd87-0c62-45b6-8d96-47954d7bf962'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:15.106035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-960abd87-0c62-45b6-8d96-47954d7bf962 admin admin] The token cache is being invalidate because project 3eb23982aa2243d3afb8827bd9150e4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:15.106516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-960abd87-0c62-45b6-8d96-47954d7bf962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:15.109182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-960abd87-0c62-45b6-8d96-47954d7bf962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:15.111410 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 922/1848] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:14 2022] DELETE /identity/v3/projects/3eb23982aa2243d3afb8827bd9150e4a => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:15.119196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca557ca-0780-44f0-b670-fcf78a950a8c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:15.119708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca557ca-0780-44f0-b670-fcf78a950a8c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:15.120208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca557ca-0780-44f0-b670-fcf78a950a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:15.180564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-8ca557ca-0780-44f0-b670-fcf78a950a8c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:15.184029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8ca557ca-0780-44f0-b670-fcf78a950a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:15.317224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 927/1849] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:15.329466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7018f3-b639-4945-a88b-fe3f8ba93572 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:15.332549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7018f3-b639-4945-a88b-fe3f8ba93572 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:15.332549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7018f3-b639-4945-a88b-fe3f8ba93572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:15.368620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-bf7018f3-b639-4945-a88b-fe3f8ba93572 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:15.371168 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bf7018f3-b639-4945-a88b-fe3f8ba93572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:15.398044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 923/1850] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:15.402569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca557ca-0780-44f0-b670-fcf78a950a8c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:15.415505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca557ca-0780-44f0-b670-fcf78a950a8c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:15.427619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ca89f-42bc-436e-a5e8-b0eb94bedf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:15.428776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ca89f-42bc-436e-a5e8-b0eb94bedf98 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:15.429277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ca89f-42bc-436e-a5e8-b0eb94bedf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:15.430012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ca89f-42bc-436e-a5e8-b0eb94bedf98 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:15.431577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ca89f-42bc-436e-a5e8-b0eb94bedf98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:05:15.431897 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ca89f-42bc-436e-a5e8-b0eb94bedf98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:15.433781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ca89f-42bc-436e-a5e8-b0eb94bedf98 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:15.453148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 928/1851] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:05:15.457455 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7018f3-b639-4945-a88b-fe3f8ba93572 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:15.471821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7018f3-b639-4945-a88b-fe3f8ba93572 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:15.478442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22482c1b-a5d2-401a-9850-c00abcadacb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:15.479310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22482c1b-a5d2-401a-9850-c00abcadacb7 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:15.479710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22482c1b-a5d2-401a-9850-c00abcadacb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:15.480148 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22482c1b-a5d2-401a-9850-c00abcadacb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:15.481016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22482c1b-a5d2-401a-9850-c00abcadacb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:15.484839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22482c1b-a5d2-401a-9850-c00abcadacb7 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:15.561052 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:15.582880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ca89f-42bc-436e-a5e8-b0eb94bedf98 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:15.586708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e31ca89f-42bc-436e-a5e8-b0eb94bedf98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:15.604035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ca89f-42bc-436e-a5e8-b0eb94bedf98 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:15.611677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:15.612076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:15.612541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:15.612918 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] PATH_INFO: `/v3/projects/5d8b1ca2b776409eb37c94a44cc25dd4` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:15.614818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d8b1ca2b776409eb37c94a44cc25dd4)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:15.616976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:15.630608 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 924/1852] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:15 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:15.635714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22482c1b-a5d2-401a-9850-c00abcadacb7 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:15.764112 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22482c1b-a5d2-401a-9850-c00abcadacb7 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:15.769519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606adcd-6992-4827-9009-82cc5f872ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:15.770224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606adcd-6992-4827-9009-82cc5f872ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:15.770544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606adcd-6992-4827-9009-82cc5f872ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:15.770957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606adcd-6992-4827-9009-82cc5f872ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:15.772070 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606adcd-6992-4827-9009-82cc5f872ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:15.774511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606adcd-6992-4827-9009-82cc5f872ff5 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:15.810317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d8b1ca2b776409eb37c94a44cc25dd4', 'request_id': 'req-ea39dc40-9bdc-4661-a252-ec4856100d44'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:15.822486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d8b1ca2b776409eb37c94a44cc25dd4', 'request_id': 'req-ea39dc40-9bdc-4661-a252-ec4856100d44'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:15.832509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d8b1ca2b776409eb37c94a44cc25dd4', 'request_id': 'req-ea39dc40-9bdc-4661-a252-ec4856100d44'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:15.941382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 925/1853] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:15 2022] POST /identity/v3/users => generated 293 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:15.945830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606adcd-6992-4827-9009-82cc5f872ff5 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:15.958138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606adcd-6992-4827-9009-82cc5f872ff5 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:15.975209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] The token cache is being invalidate because project 5d8b1ca2b776409eb37c94a44cc25dd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:15.975797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:15.976721 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:15.979031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 929/1854] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:15 2022] DELETE /identity/v3/projects/5d8b1ca2b776409eb37c94a44cc25dd4 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:15.983938 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:15.986079 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:16.036291 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5311449-480f-4818-b129-ef91b24542eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:16.036991 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5311449-480f-4818-b129-ef91b24542eb admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:16.037332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5311449-480f-4818-b129-ef91b24542eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:16.037604 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5311449-480f-4818-b129-ef91b24542eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:16.038368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5311449-480f-4818-b129-ef91b24542eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:16.040253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5311449-480f-4818-b129-ef91b24542eb admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:16.052032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 926/1855] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:16.056458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5311449-480f-4818-b129-ef91b24542eb admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:16.058184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c5311449-480f-4818-b129-ef91b24542eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:16.072567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39dc40-9bdc-4661-a252-ec4856100d44 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:16.078641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5311449-480f-4818-b129-ef91b24542eb admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:16.080832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:16.085593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:16.085593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:16.085593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] PATH_INFO: `/v3/users/51af43676ba2410ea8de4e660ffe597f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:16.085593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] RBAC: Authorizing `identity:delete_user(user_id=51af43676ba2410ea8de4e660ffe597f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:16.085958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c4652-3435-4bd1-bd55-0d9017f17372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:16.085958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c4652-3435-4bd1-bd55-0d9017f17372 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:16.085958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c4652-3435-4bd1-bd55-0d9017f17372 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:16.085958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c4652-3435-4bd1-bd55-0d9017f17372 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:16.086247 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c4652-3435-4bd1-bd55-0d9017f17372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:16.086469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:16.088026 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c4652-3435-4bd1-bd55-0d9017f17372 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:16.105218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 927/1856] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:16.106787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c4652-3435-4bd1-bd55-0d9017f17372 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:16.121154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c4652-3435-4bd1-bd55-0d9017f17372 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:16.128426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e286c-fff4-4ad9-937d-059b8054dc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:16.133661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e286c-fff4-4ad9-937d-059b8054dc91 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:16.133661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e286c-fff4-4ad9-937d-059b8054dc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:16.133661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e286c-fff4-4ad9-937d-059b8054dc91 admin admin] PATH_INFO: `/v3/projects/808e941babe345a1bed7bcd020f9652d/users/2a045745710442b098058c68c3689711/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:16.159328 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e286c-fff4-4ad9-937d-059b8054dc91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=808e941babe345a1bed7bcd020f9652d, user_id=2a045745710442b098058c68c3689711, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:16.162175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e286c-fff4-4ad9-937d-059b8054dc91 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:16.294653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 928/1857] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:16 2022] PUT /identity/v3/projects/808e941babe345a1bed7bcd020f9652d/users/2a045745710442b098058c68c3689711/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:16.308463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c23ab3-01ef-4eb5-8628-c33811ceda97 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:16.309019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c23ab3-01ef-4eb5-8628-c33811ceda97 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:16.309434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c23ab3-01ef-4eb5-8628-c33811ceda97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:16.362308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e3c23ab3-01ef-4eb5-8628-c33811ceda97 None None] MFA Rules not processed for user `2a045745710442b098058c68c3689711`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:16.364296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e3c23ab3-01ef-4eb5-8628-c33811ceda97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:16.372245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51af43676ba2410ea8de4e660ffe597f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:16.373146 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51af43676ba2410ea8de4e660ffe597f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:16.386746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51af43676ba2410ea8de4e660ffe597f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:16.406684 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51af43676ba2410ea8de4e660ffe597f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:16.433121 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:16.464678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 929/1858] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:16.470563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c23ab3-01ef-4eb5-8628-c33811ceda97 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:16.491380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51af43676ba2410ea8de4e660ffe597f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:16.500696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 930/1859] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:15 2022] DELETE /identity/v3/users/51af43676ba2410ea8de4e660ffe597f => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:16.581641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c23ab3-01ef-4eb5-8628-c33811ceda97 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:16.631698 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:16.732029 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7efc5-49af-4e9a-b7f2-7ac38e6e867e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:16.733093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7efc5-49af-4e9a-b7f2-7ac38e6e867e admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:16.734043 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7efc5-49af-4e9a-b7f2-7ac38e6e867e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:16.738856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7efc5-49af-4e9a-b7f2-7ac38e6e867e admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:16.740010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7efc5-49af-4e9a-b7f2-7ac38e6e867e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:16.743240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7efc5-49af-4e9a-b7f2-7ac38e6e867e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:16.892828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 930/1860] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:16 2022] POST /identity/v3/projects => generated 383 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:05:16.897982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:16.900707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:16.920367 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db567fff-600d-4dd3-a4a8-5ec274b4e9da admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:17.032759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4e93d-f72d-4fde-913f-9553d41d8c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:17.033559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4e93d-f72d-4fde-913f-9553d41d8c9d admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:17.034078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4e93d-f72d-4fde-913f-9553d41d8c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:17.034532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4e93d-f72d-4fde-913f-9553d41d8c9d admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:17.035404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4e93d-f72d-4fde-913f-9553d41d8c9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:17.039196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4e93d-f72d-4fde-913f-9553d41d8c9d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:17.152502 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:17.196719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7efc5-49af-4e9a-b7f2-7ac38e6e867e admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:17.199194 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-dba7efc5-49af-4e9a-b7f2-7ac38e6e867e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:17.200087 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 931/1861] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:16 2022] POST /identity/v3/users => generated 298 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:17.205816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4e93d-f72d-4fde-913f-9553d41d8c9d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:17.219862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7efc5-49af-4e9a-b7f2-7ac38e6e867e admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:17.225075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4e93d-f72d-4fde-913f-9553d41d8c9d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:17.227317 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894dcbc-e190-43c3-bd41-48a452967036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:17.228448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894dcbc-e190-43c3-bd41-48a452967036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:17.228738 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894dcbc-e190-43c3-bd41-48a452967036 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:17.228972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894dcbc-e190-43c3-bd41-48a452967036 admin admin] PATH_INFO: `/v3/projects/5ba1c78d6bdb4faebe9ce40ec6a3b530` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:17.230240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894dcbc-e190-43c3-bd41-48a452967036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba1c78d6bdb4faebe9ce40ec6a3b530)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:17.233168 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894dcbc-e190-43c3-bd41-48a452967036 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:17.242832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:17.244647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db667a6d-7479-45c4-bf05-bf1b495fd2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:17.246428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db667a6d-7479-45c4-bf05-bf1b495fd2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:17.246911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db667a6d-7479-45c4-bf05-bf1b495fd2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:17.247548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db667a6d-7479-45c4-bf05-bf1b495fd2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:17.248965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db667a6d-7479-45c4-bf05-bf1b495fd2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:17.251180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db667a6d-7479-45c4-bf05-bf1b495fd2c0 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:17.266540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 932/1862] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:17.273776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db667a6d-7479-45c4-bf05-bf1b495fd2c0 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:17.286146 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db667a6d-7479-45c4-bf05-bf1b495fd2c0 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:17.293185 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107bede-0554-4641-816f-aea01397a834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:17.295192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107bede-0554-4641-816f-aea01397a834 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:17.295192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107bede-0554-4641-816f-aea01397a834 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:17.295192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107bede-0554-4641-816f-aea01397a834 admin admin] PATH_INFO: `/v3/projects/21053db796e4440bb8901bca240edacd/users/064180adf52d40b2bd5087cce1fcbdec/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:17.315473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107bede-0554-4641-816f-aea01397a834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21053db796e4440bb8901bca240edacd, user_id=064180adf52d40b2bd5087cce1fcbdec, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:17.318419 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107bede-0554-4641-816f-aea01397a834 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:17.434630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 933/1863] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:17 2022] PUT /identity/v3/projects/21053db796e4440bb8901bca240edacd/users/064180adf52d40b2bd5087cce1fcbdec/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:17.442356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107bede-0554-4641-816f-aea01397a834 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:17.444212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d894dcbc-e190-43c3-bd41-48a452967036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba1c78d6bdb4faebe9ce40ec6a3b530', 'request_id': 'req-d894dcbc-e190-43c3-bd41-48a452967036'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:17.554832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107bede-0554-4641-816f-aea01397a834 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:17.557276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d894dcbc-e190-43c3-bd41-48a452967036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba1c78d6bdb4faebe9ce40ec6a3b530', 'request_id': 'req-d894dcbc-e190-43c3-bd41-48a452967036'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:17.573950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d894dcbc-e190-43c3-bd41-48a452967036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba1c78d6bdb4faebe9ce40ec6a3b530', 'request_id': 'req-d894dcbc-e190-43c3-bd41-48a452967036'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:17.598209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d894dcbc-e190-43c3-bd41-48a452967036 admin admin] The token cache is being invalidate because project 5ba1c78d6bdb4faebe9ce40ec6a3b530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:17.598557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d894dcbc-e190-43c3-bd41-48a452967036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:17.599858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d894dcbc-e190-43c3-bd41-48a452967036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:17.606922 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 931/1864] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:17 2022] DELETE /identity/v3/projects/5ba1c78d6bdb4faebe9ce40ec6a3b530 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:17.609128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfafc1-f8ba-44b5-bc3d-932258609c93 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:17.609437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfafc1-f8ba-44b5-bc3d-932258609c93 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:17.609974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfafc1-f8ba-44b5-bc3d-932258609c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:17.635466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-f2cfafc1-f8ba-44b5-bc3d-932258609c93 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:17.636986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f2cfafc1-f8ba-44b5-bc3d-932258609c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:17.649765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b07abd-08d3-4230-a458-5165505a06df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:17.650927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b07abd-08d3-4230-a458-5165505a06df admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:17.651616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b07abd-08d3-4230-a458-5165505a06df admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:17.652046 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b07abd-08d3-4230-a458-5165505a06df admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:17.652996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b07abd-08d3-4230-a458-5165505a06df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:17.654878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b07abd-08d3-4230-a458-5165505a06df admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:17.673121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 934/1865] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:17.674616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b07abd-08d3-4230-a458-5165505a06df admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:17.677195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-71b07abd-08d3-4230-a458-5165505a06df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:17.694359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b07abd-08d3-4230-a458-5165505a06df admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:17.702113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85ca1f-c227-4c47-8b3b-89be774c49fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:17.705336 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85ca1f-c227-4c47-8b3b-89be774c49fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:17.705629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85ca1f-c227-4c47-8b3b-89be774c49fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:17.705917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85ca1f-c227-4c47-8b3b-89be774c49fe admin admin] PATH_INFO: `/v3/projects/21053db796e4440bb8901bca240edacd/users/064180adf52d40b2bd5087cce1fcbdec/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:17.708234 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85ca1f-c227-4c47-8b3b-89be774c49fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=21053db796e4440bb8901bca240edacd, user_id=064180adf52d40b2bd5087cce1fcbdec, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:17.711147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85ca1f-c227-4c47-8b3b-89be774c49fe admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:17.727535 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 935/1866] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:17 2022] PUT /identity/v3/projects/21053db796e4440bb8901bca240edacd/users/064180adf52d40b2bd5087cce1fcbdec/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:17.751162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 932/1867] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:17.751638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94552afd-2b04-4a8d-85e0-d89fcf23d28f None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:17.751638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94552afd-2b04-4a8d-85e0-d89fcf23d28f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:17.751638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94552afd-2b04-4a8d-85e0-d89fcf23d28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:17.751906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45807723-b379-48e2-8ce5-f1aed7c51133 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:17.751906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45807723-b379-48e2-8ce5-f1aed7c51133 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:17.751906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45807723-b379-48e2-8ce5-f1aed7c51133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:17.762355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:17.784606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-45807723-b379-48e2-8ce5-f1aed7c51133 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:17.786797 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-45807723-b379-48e2-8ce5-f1aed7c51133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:17.797559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-94552afd-2b04-4a8d-85e0-d89fcf23d28f None None] MFA Rules not processed for user `064180adf52d40b2bd5087cce1fcbdec`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:17.799113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-94552afd-2b04-4a8d-85e0-d89fcf23d28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:17.808021 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:17.892916 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 933/1868] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:17.901120 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45807723-b379-48e2-8ce5-f1aed7c51133 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:17.918239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45807723-b379-48e2-8ce5-f1aed7c51133 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:17.918843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 936/1869] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:17.930351 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598256c5-77cd-4514-804b-e6e733098542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:17.931811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598256c5-77cd-4514-804b-e6e733098542 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:17.932264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598256c5-77cd-4514-804b-e6e733098542 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:17.932549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598256c5-77cd-4514-804b-e6e733098542 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:17.933371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598256c5-77cd-4514-804b-e6e733098542 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:05:17.934233 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598256c5-77cd-4514-804b-e6e733098542 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:17.935532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598256c5-77cd-4514-804b-e6e733098542 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:17.939849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94552afd-2b04-4a8d-85e0-d89fcf23d28f None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:17.944657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-94552afd-2b04-4a8d-85e0-d89fcf23d28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:17.953471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 934/1870] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:05:17.953471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:17.964208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598256c5-77cd-4514-804b-e6e733098542 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:17.976973 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598256c5-77cd-4514-804b-e6e733098542 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:17.982727 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b9f7f-7fbb-41af-aa6c-c50eeeaccdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:17.983359 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9f7f-7fbb-41af-aa6c-c50eeeaccdb0 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:17.984409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9f7f-7fbb-41af-aa6c-c50eeeaccdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:17.984897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9f7f-7fbb-41af-aa6c-c50eeeaccdb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:17.985821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b9f7f-7fbb-41af-aa6c-c50eeeaccdb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:17.988023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b9f7f-7fbb-41af-aa6c-c50eeeaccdb0 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:18.037365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94552afd-2b04-4a8d-85e0-d89fcf23d28f None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:18.047167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e48558-a970-484d-81cc-b0c1191d166d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:18.047960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e48558-a970-484d-81cc-b0c1191d166d service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:18.048333 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e48558-a970-484d-81cc-b0c1191d166d service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:18.048712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e48558-a970-484d-81cc-b0c1191d166d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:18.051105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d1e48558-a970-484d-81cc-b0c1191d166d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:18.105001 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 935/1871] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:17 2022] POST /identity/v3/projects => generated 379 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:18.109276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b9f7f-7fbb-41af-aa6c-c50eeeaccdb0 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:18.166997 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e48558-a970-484d-81cc-b0c1191d166d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:18.168479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e48558-a970-484d-81cc-b0c1191d166d service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:18.178718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:18.193796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 937/1872] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:05:18.215591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b9f7f-7fbb-41af-aa6c-c50eeeaccdb0 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:18.228197 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7b282-2ce0-400e-8107-64a3512bdecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:18.229098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7b282-2ce0-400e-8107-64a3512bdecf admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:18.229494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7b282-2ce0-400e-8107-64a3512bdecf admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:18.229881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7b282-2ce0-400e-8107-64a3512bdecf admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:18.230918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7b282-2ce0-400e-8107-64a3512bdecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:18.242377 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7b282-2ce0-400e-8107-64a3512bdecf admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:18.281662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:18.387724 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 936/1873] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:18 2022] POST /identity/v3/users => generated 290 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:18.393010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e48558-a970-484d-81cc-b0c1191d166d service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:18.408062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e48558-a970-484d-81cc-b0c1191d166d service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:18.416210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3eb36-e5e1-48b1-a193-d801b8236f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:18.417091 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3eb36-e5e1-48b1-a193-d801b8236f93 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:18.417394 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3eb36-e5e1-48b1-a193-d801b8236f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:18.417732 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3eb36-e5e1-48b1-a193-d801b8236f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:18.418539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3eb36-e5e1-48b1-a193-d801b8236f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:18.420162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3eb36-e5e1-48b1-a193-d801b8236f93 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:18.440327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 938/1874] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:18.441953 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7b282-2ce0-400e-8107-64a3512bdecf admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:18.454852 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7b282-2ce0-400e-8107-64a3512bdecf admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:18.459941 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59461d23-f7f2-46f2-a19e-7a26934f696b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:18.460743 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59461d23-f7f2-46f2-a19e-7a26934f696b admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:18.461033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59461d23-f7f2-46f2-a19e-7a26934f696b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:18.461428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59461d23-f7f2-46f2-a19e-7a26934f696b admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:18.462305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59461d23-f7f2-46f2-a19e-7a26934f696b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:18.463802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59461d23-f7f2-46f2-a19e-7a26934f696b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:18.476960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 937/1875] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:18.484495 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3eb36-e5e1-48b1-a193-d801b8236f93 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:18.495791 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3eb36-e5e1-48b1-a193-d801b8236f93 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:18.501859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae89b7-3ae4-40ba-af74-0a0fb16e310c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:18.502814 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae89b7-3ae4-40ba-af74-0a0fb16e310c admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:18.503098 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae89b7-3ae4-40ba-af74-0a0fb16e310c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:18.503369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae89b7-3ae4-40ba-af74-0a0fb16e310c admin admin] PATH_INFO: `/v3/projects/5ccd44c3a1aa4edc88b261496cf58d6b/users/77d41b1d574d4ace8fe94ad890183c8e/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:18.509985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:18.524140 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae89b7-3ae4-40ba-af74-0a0fb16e310c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ccd44c3a1aa4edc88b261496cf58d6b, user_id=77d41b1d574d4ace8fe94ad890183c8e, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:18.527251 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae89b7-3ae4-40ba-af74-0a0fb16e310c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:18.585677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:18.661700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 939/1876] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:18 2022] PUT /identity/v3/projects/5ccd44c3a1aa4edc88b261496cf58d6b/users/77d41b1d574d4ace8fe94ad890183c8e/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:18.674160 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c33ba-a521-48b9-8082-84b27e70155f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:18.674160 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c33ba-a521-48b9-8082-84b27e70155f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:18.674160 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c33ba-a521-48b9-8082-84b27e70155f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:18.705359 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-492c33ba-a521-48b9-8082-84b27e70155f None None] MFA Rules not processed for user `77d41b1d574d4ace8fe94ad890183c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:18.706555 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-492c33ba-a521-48b9-8082-84b27e70155f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:18.784742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 938/1877] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:18.789647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae89b7-3ae4-40ba-af74-0a0fb16e310c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:18.887211 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae89b7-3ae4-40ba-af74-0a0fb16e310c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:18.898297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b42115-5d9d-4bc7-892f-0357638211e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:18.899199 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b42115-5d9d-4bc7-892f-0357638211e0 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:18.899493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b42115-5d9d-4bc7-892f-0357638211e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:18.899833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b42115-5d9d-4bc7-892f-0357638211e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:18.900961 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b42115-5d9d-4bc7-892f-0357638211e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:18.903921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b42115-5d9d-4bc7-892f-0357638211e0 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:19.072876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 940/1878] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:18 2022] POST /identity/v3/projects => generated 377 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:19.077068 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c33ba-a521-48b9-8082-84b27e70155f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:19.092394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c33ba-a521-48b9-8082-84b27e70155f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:19.181630 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72638c-831c-441b-9fde-365647c02ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:19.182385 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72638c-831c-441b-9fde-365647c02ecc admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:19.182685 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72638c-831c-441b-9fde-365647c02ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:19.183002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72638c-831c-441b-9fde-365647c02ecc admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:19.183722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72638c-831c-441b-9fde-365647c02ecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:19.189017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72638c-831c-441b-9fde-365647c02ecc admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:19.208456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:19.304850 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:19.309240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 939/1879] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:19 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:19.314401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b42115-5d9d-4bc7-892f-0357638211e0 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:19.327174 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b42115-5d9d-4bc7-892f-0357638211e0 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:19.337292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4980af1-c360-47af-b817-5b3d0cc4ab58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:19.337292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4980af1-c360-47af-b817-5b3d0cc4ab58 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:19.337292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4980af1-c360-47af-b817-5b3d0cc4ab58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:19.337292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4980af1-c360-47af-b817-5b3d0cc4ab58 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:19.338121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4980af1-c360-47af-b817-5b3d0cc4ab58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:19.338780 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4980af1-c360-47af-b817-5b3d0cc4ab58 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:19.359969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 941/1880] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:19.362071 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72638c-831c-441b-9fde-365647c02ecc admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:19.381329 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72638c-831c-441b-9fde-365647c02ecc admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:19.389631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195f8a8-c605-403d-a3e4-cf475f6036fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:19.391103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195f8a8-c605-403d-a3e4-cf475f6036fb admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:19.391772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195f8a8-c605-403d-a3e4-cf475f6036fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:19.392189 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195f8a8-c605-403d-a3e4-cf475f6036fb admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:19.394003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195f8a8-c605-403d-a3e4-cf475f6036fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:19.396161 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195f8a8-c605-403d-a3e4-cf475f6036fb admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:19.417232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 940/1881] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:19.423755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4980af1-c360-47af-b817-5b3d0cc4ab58 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:19.439799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4980af1-c360-47af-b817-5b3d0cc4ab58 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:19.445788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412a046-c695-426c-aa61-32b8a89c221e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:19.447451 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412a046-c695-426c-aa61-32b8a89c221e admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:19.448139 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412a046-c695-426c-aa61-32b8a89c221e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:19.448755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412a046-c695-426c-aa61-32b8a89c221e admin admin] PATH_INFO: `/v3/projects/2e41c6e14ac5479094a374595ab6d93e/users/910595392b9f4810af96fd73bc60314a/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:19.484995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412a046-c695-426c-aa61-32b8a89c221e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e41c6e14ac5479094a374595ab6d93e, user_id=910595392b9f4810af96fd73bc60314a, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:19.488076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412a046-c695-426c-aa61-32b8a89c221e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:19.618467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 942/1882] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:19 2022] PUT /identity/v3/projects/2e41c6e14ac5479094a374595ab6d93e/users/910595392b9f4810af96fd73bc60314a/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:19.638731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3926431-8e49-4875-8d69-c70ee88c00f8 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:19.639236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3926431-8e49-4875-8d69-c70ee88c00f8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:19.639651 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3926431-8e49-4875-8d69-c70ee88c00f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:19.679197 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c3926431-8e49-4875-8d69-c70ee88c00f8 None None] MFA Rules not processed for user `910595392b9f4810af96fd73bc60314a`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:19.680697 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c3926431-8e49-4875-8d69-c70ee88c00f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:19.769431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 941/1883] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:19.772584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412a046-c695-426c-aa61-32b8a89c221e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:19.845662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412a046-c695-426c-aa61-32b8a89c221e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:19.853024 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15501f1a-bfc3-43dd-b4b5-4fb7f78d23f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:19.853767 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15501f1a-bfc3-43dd-b4b5-4fb7f78d23f9 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:19.854060 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15501f1a-bfc3-43dd-b4b5-4fb7f78d23f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:19.854477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15501f1a-bfc3-43dd-b4b5-4fb7f78d23f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:19.855195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15501f1a-bfc3-43dd-b4b5-4fb7f78d23f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:19.857411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15501f1a-bfc3-43dd-b4b5-4fb7f78d23f9 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:20.161973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 943/1884] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:19 2022] POST /identity/v3/projects => generated 379 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:20.167017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3926431-8e49-4875-8d69-c70ee88c00f8 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:20.179853 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3926431-8e49-4875-8d69-c70ee88c00f8 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:20.267236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34313d-35bc-469a-a787-15da0a3c8f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:20.268019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34313d-35bc-469a-a787-15da0a3c8f1e admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:20.268421 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34313d-35bc-469a-a787-15da0a3c8f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:20.268612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34313d-35bc-469a-a787-15da0a3c8f1e admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:20.269294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34313d-35bc-469a-a787-15da0a3c8f1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:20.272460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34313d-35bc-469a-a787-15da0a3c8f1e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:20.396063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 942/1885] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:20 2022] POST /identity/v3/users => generated 296 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:20.400086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15501f1a-bfc3-43dd-b4b5-4fb7f78d23f9 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:20.411461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15501f1a-bfc3-43dd-b4b5-4fb7f78d23f9 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:20.419761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceffa41-b67e-4489-a27a-feed217cca72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:20.420501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceffa41-b67e-4489-a27a-feed217cca72 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:20.420776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceffa41-b67e-4489-a27a-feed217cca72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:20.421641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceffa41-b67e-4489-a27a-feed217cca72 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:20.422416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceffa41-b67e-4489-a27a-feed217cca72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:20.424036 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceffa41-b67e-4489-a27a-feed217cca72 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:20.443850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 944/1886] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:20.448789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34313d-35bc-469a-a787-15da0a3c8f1e admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:20.461628 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34313d-35bc-469a-a787-15da0a3c8f1e admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:20.468293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b581ef-0d0b-4acd-a7c6-cfe8295471bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:20.468293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b581ef-0d0b-4acd-a7c6-cfe8295471bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:20.468575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b581ef-0d0b-4acd-a7c6-cfe8295471bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:20.468737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b581ef-0d0b-4acd-a7c6-cfe8295471bb admin admin] PATH_INFO: `/v3/projects/eff99571dd3849168370f92b2690bdee/users/52fb0c35617147d89fc2b9564cddd7a4/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:20.495249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b581ef-0d0b-4acd-a7c6-cfe8295471bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff99571dd3849168370f92b2690bdee, user_id=52fb0c35617147d89fc2b9564cddd7a4, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:20.500724 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b581ef-0d0b-4acd-a7c6-cfe8295471bb admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:20.618270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 943/1887] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:20 2022] PUT /identity/v3/projects/eff99571dd3849168370f92b2690bdee/users/52fb0c35617147d89fc2b9564cddd7a4/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:20.622730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceffa41-b67e-4489-a27a-feed217cca72 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:20.650248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceffa41-b67e-4489-a27a-feed217cca72 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:20.726253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:20.750737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71d044-a7b8-4dac-a750-efd5d4b17fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:20.751549 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71d044-a7b8-4dac-a750-efd5d4b17fdc admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:20.751839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71d044-a7b8-4dac-a750-efd5d4b17fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:20.752206 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71d044-a7b8-4dac-a750-efd5d4b17fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:20.752968 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71d044-a7b8-4dac-a750-efd5d4b17fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:20.754895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71d044-a7b8-4dac-a750-efd5d4b17fdc admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:20.771805 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 945/1888] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:20.779216 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b581ef-0d0b-4acd-a7c6-cfe8295471bb admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:20.793156 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b581ef-0d0b-4acd-a7c6-cfe8295471bb admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:20.799849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0df262-8571-4d81-8ad7-504babb5472e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:20.800750 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0df262-8571-4d81-8ad7-504babb5472e admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:20.801039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0df262-8571-4d81-8ad7-504babb5472e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:20.801379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0df262-8571-4d81-8ad7-504babb5472e admin admin] PATH_INFO: `/v3/projects/eff99571dd3849168370f92b2690bdee/users/52fb0c35617147d89fc2b9564cddd7a4/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:20.804211 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0df262-8571-4d81-8ad7-504babb5472e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff99571dd3849168370f92b2690bdee, user_id=52fb0c35617147d89fc2b9564cddd7a4, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:20.807221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0df262-8571-4d81-8ad7-504babb5472e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:20.821384 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 944/1889] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:20 2022] PUT /identity/v3/projects/eff99571dd3849168370f92b2690bdee/users/52fb0c35617147d89fc2b9564cddd7a4/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:20.833160 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f09fb2-3e73-4423-a817-1712ff6676a6 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:20.833880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f09fb2-3e73-4423-a817-1712ff6676a6 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:20.833880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f09fb2-3e73-4423-a817-1712ff6676a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:20.866289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-11f09fb2-3e73-4423-a817-1712ff6676a6 None None] MFA Rules not processed for user `52fb0c35617147d89fc2b9564cddd7a4`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:20.867567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-11f09fb2-3e73-4423-a817-1712ff6676a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:20.947996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 946/1890] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:20.960047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0df262-8571-4d81-8ad7-504babb5472e admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:20.970432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:21.027340 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0df262-8571-4d81-8ad7-504babb5472e admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:21.031881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce89e2-0221-466c-99e5-8c5fc78db803 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:21.032618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce89e2-0221-466c-99e5-8c5fc78db803 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:21.032876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce89e2-0221-466c-99e5-8c5fc78db803 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:21.033205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce89e2-0221-466c-99e5-8c5fc78db803 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:21.056319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce89e2-0221-466c-99e5-8c5fc78db803 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:21.056319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce89e2-0221-466c-99e5-8c5fc78db803 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:21.061310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 945/1891] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:05:21.070240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:21.376182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f09fb2-3e73-4423-a817-1712ff6676a6 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:21.395876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f09fb2-3e73-4423-a817-1712ff6676a6 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:21.415764 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb21e7f-6eca-47a0-81a2-e69f97902b02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:21.416497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb21e7f-6eca-47a0-81a2-e69f97902b02 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:21.416783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb21e7f-6eca-47a0-81a2-e69f97902b02 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:21.417092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb21e7f-6eca-47a0-81a2-e69f97902b02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:21.510807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb21e7f-6eca-47a0-81a2-e69f97902b02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:21.512969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb21e7f-6eca-47a0-81a2-e69f97902b02 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:21.528209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:21.533609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 947/1892] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:21.844725 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc71481-337b-40d6-b7b4-25ffd6951b20 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:21.845289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc71481-337b-40d6-b7b4-25ffd6951b20 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:21.845617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc71481-337b-40d6-b7b4-25ffd6951b20 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:21.847733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 946/1893] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 19:05:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:05:21.861712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d43e4-af03-4301-8f8a-f0d292776d0c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:21.862205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d43e4-af03-4301-8f8a-f0d292776d0c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:21.862482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d43e4-af03-4301-8f8a-f0d292776d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:21.892229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-880d43e4-af03-4301-8f8a-f0d292776d0c None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:21.894359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-880d43e4-af03-4301-8f8a-f0d292776d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:22.041023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 948/1894] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 19:05:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:22.084442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc71481-337b-40d6-b7b4-25ffd6951b20 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:22.086416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-efc71481-337b-40d6-b7b4-25ffd6951b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:22.108114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc71481-337b-40d6-b7b4-25ffd6951b20 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:22.113982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae623c4c-b96c-4e88-9aab-a75bfa33769f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:22.114754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae623c4c-b96c-4e88-9aab-a75bfa33769f service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:22.120429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae623c4c-b96c-4e88-9aab-a75bfa33769f service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:22.120429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae623c4c-b96c-4e88-9aab-a75bfa33769f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:22.127684 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae623c4c-b96c-4e88-9aab-a75bfa33769f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:22.129320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae623c4c-b96c-4e88-9aab-a75bfa33769f service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:22.142236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 947/1895] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:27.021094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d43e4-af03-4301-8f8a-f0d292776d0c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:27.022769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-880d43e4-af03-4301-8f8a-f0d292776d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:27.148781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d43e4-af03-4301-8f8a-f0d292776d0c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:27.153861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:27.154896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:27.155719 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:27.155964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] PATH_INFO: `/v3/users/a5e50c563c9741c3a06b474f8636eb99` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:27.157454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5e50c563c9741c3a06b474f8636eb99)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:27.159586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:27.165987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:27.441127 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5e50c563c9741c3a06b474f8636eb99'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:27.441900 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5e50c563c9741c3a06b474f8636eb99'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:27.450193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5e50c563c9741c3a06b474f8636eb99'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:27.466787 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5e50c563c9741c3a06b474f8636eb99'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:27.564780 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5e50c563c9741c3a06b474f8636eb99'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:27.577131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 949/1896] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:27 2022] DELETE /identity/v3/users/a5e50c563c9741c3a06b474f8636eb99 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:27.579281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb706ca2-bffd-41df-b03f-ecb390f58117 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:27.580260 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb706ca2-bffd-41df-b03f-ecb390f58117 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:27.580429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb706ca2-bffd-41df-b03f-ecb390f58117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:27.612578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-bb706ca2-bffd-41df-b03f-ecb390f58117 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:27.615074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bb706ca2-bffd-41df-b03f-ecb390f58117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:27.690565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 948/1897] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:27.700659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:27.706017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:27.812394 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e439434-85b6-4c5e-8f1e-ca545ddd7217 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:27.817086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e8f89-f0b7-4d3f-a784-b08c28c566c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:27.818225 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e8f89-f0b7-4d3f-a784-b08c28c566c8 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:27.818683 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e8f89-f0b7-4d3f-a784-b08c28c566c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:27.819188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e8f89-f0b7-4d3f-a784-b08c28c566c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:27.837982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e8f89-f0b7-4d3f-a784-b08c28c566c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:27.840566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e8f89-f0b7-4d3f-a784-b08c28c566c8 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:27.855338 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 950/1898] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:28.685650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb706ca2-bffd-41df-b03f-ecb390f58117 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:28.687991 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bb706ca2-bffd-41df-b03f-ecb390f58117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:28.702296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e8f89-f0b7-4d3f-a784-b08c28c566c8 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:28.703068 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb706ca2-bffd-41df-b03f-ecb390f58117 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:28.703989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-2a6e8f89-f0b7-4d3f-a784-b08c28c566c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:28.716542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:28.718111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:28.718673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:28.719648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] PATH_INFO: `/v3/users/77d41b1d574d4ace8fe94ad890183c8e` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:28.722682 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77d41b1d574d4ace8fe94ad890183c8e)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:28.726662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:28.728129 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e8f89-f0b7-4d3f-a784-b08c28c566c8 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:28.744214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669dba27-5225-402c-bf6c-065b71473c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:28.745225 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dba27-5225-402c-bf6c-065b71473c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:28.745512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dba27-5225-402c-bf6c-065b71473c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:28.746601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dba27-5225-402c-bf6c-065b71473c92 admin admin] PATH_INFO: `/v3/projects/dc9aa94d422144f786b359a94c0154e6` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:28.748496 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669dba27-5225-402c-bf6c-065b71473c92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc9aa94d422144f786b359a94c0154e6)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:28.754686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669dba27-5225-402c-bf6c-065b71473c92 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:29.083389 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77d41b1d574d4ace8fe94ad890183c8e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.084568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77d41b1d574d4ace8fe94ad890183c8e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.085210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77d41b1d574d4ace8fe94ad890183c8e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.092487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-669dba27-5225-402c-bf6c-065b71473c92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc9aa94d422144f786b359a94c0154e6', 'request_id': 'req-669dba27-5225-402c-bf6c-065b71473c92'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.104448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-669dba27-5225-402c-bf6c-065b71473c92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc9aa94d422144f786b359a94c0154e6', 'request_id': 'req-669dba27-5225-402c-bf6c-065b71473c92'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.113360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-669dba27-5225-402c-bf6c-065b71473c92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc9aa94d422144f786b359a94c0154e6', 'request_id': 'req-669dba27-5225-402c-bf6c-065b71473c92'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.163119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77d41b1d574d4ace8fe94ad890183c8e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.180083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77d41b1d574d4ace8fe94ad890183c8e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.193878 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 949/1899] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:28 2022] DELETE /identity/v3/users/77d41b1d574d4ace8fe94ad890183c8e => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:29.195184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:29.197517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:29.211420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:29.222384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-669dba27-5225-402c-bf6c-065b71473c92 admin admin] The token cache is being invalidate because project dc9aa94d422144f786b359a94c0154e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:29.222601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-669dba27-5225-402c-bf6c-065b71473c92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.223837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-669dba27-5225-402c-bf6c-065b71473c92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.241806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 951/1900] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:28 2022] DELETE /identity/v3/projects/dc9aa94d422144f786b359a94c0154e6 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:29.241806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50911c-95dc-44ac-8548-362e56af40ac None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:29.241806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50911c-95dc-44ac-8548-362e56af40ac None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:29.241806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50911c-95dc-44ac-8548-362e56af40ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:29.261244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-7c50911c-95dc-44ac-8548-362e56af40ac None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:29.262605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7c50911c-95dc-44ac-8548-362e56af40ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:29.269278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:29.276041 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe8365-dd3a-4ff9-95dc-3e78ff8eb920 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:29.281044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c5e02-f714-4210-a3e5-e69284b463b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:29.281885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c5e02-f714-4210-a3e5-e69284b463b4 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:29.284457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c5e02-f714-4210-a3e5-e69284b463b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:29.284457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c5e02-f714-4210-a3e5-e69284b463b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:29.284457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-753c5e02-f714-4210-a3e5-e69284b463b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:29.344010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 952/1901] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:29.345040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50911c-95dc-44ac-8548-362e56af40ac None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:29.346632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7c50911c-95dc-44ac-8548-362e56af40ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:29.361208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50911c-95dc-44ac-8548-362e56af40ac None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:29.367342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:29.368441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:29.368852 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:29.370988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] PATH_INFO: `/v3/users/9db0ead52a3b4c4083fc1b06aa2628c4` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:29.371240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=9db0ead52a3b4c4083fc1b06aa2628c4)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:29.374092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:29.375224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c5e02-f714-4210-a3e5-e69284b463b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:29.377189 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c5e02-f714-4210-a3e5-e69284b463b4 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:29.393882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 950/1902] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:29.395031 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c5e02-f714-4210-a3e5-e69284b463b4 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:29.396493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-753c5e02-f714-4210-a3e5-e69284b463b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:29.480159 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c5e02-f714-4210-a3e5-e69284b463b4 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:29.485401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d1a8a-a245-4134-a9f1-684c9d707fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:29.486169 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d1a8a-a245-4134-a9f1-684c9d707fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:29.486169 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d1a8a-a245-4134-a9f1-684c9d707fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:29.486657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d1a8a-a245-4134-a9f1-684c9d707fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:29.498700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d1a8a-a245-4134-a9f1-684c9d707fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:29.501092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d1a8a-a245-4134-a9f1-684c9d707fb9 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:29.516774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 951/1903] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:29.690241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9db0ead52a3b4c4083fc1b06aa2628c4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.691030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9db0ead52a3b4c4083fc1b06aa2628c4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.708711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9db0ead52a3b4c4083fc1b06aa2628c4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.732157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9db0ead52a3b4c4083fc1b06aa2628c4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.816758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9db0ead52a3b4c4083fc1b06aa2628c4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:29.827017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 953/1904] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:29 2022] DELETE /identity/v3/users/9db0ead52a3b4c4083fc1b06aa2628c4 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:30.411183 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d1a8a-a245-4134-a9f1-684c9d707fb9 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:30.416001 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-087d1a8a-a245-4134-a9f1-684c9d707fb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:30.455817 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d1a8a-a245-4134-a9f1-684c9d707fb9 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:30.542771 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:30.543782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:30.544139 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:30.544483 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] PATH_INFO: `/v3/projects/5ccd44c3a1aa4edc88b261496cf58d6b` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:30.546199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ccd44c3a1aa4edc88b261496cf58d6b)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:30.548886 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:30.562233 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:30.580749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:30.584041 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:30.614233 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bce97-e8ce-4e99-bbaf-39423b0867ea admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:30.630055 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65df0e3-a460-4e88-b113-44184c2625eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:30.636028 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65df0e3-a460-4e88-b113-44184c2625eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:30.636326 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65df0e3-a460-4e88-b113-44184c2625eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:30.636611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65df0e3-a460-4e88-b113-44184c2625eb admin admin] PATH_INFO: `/v3/projects/81ff84d4e534408e8e47b1de5a7bdbf8` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:30.642073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65df0e3-a460-4e88-b113-44184c2625eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ff84d4e534408e8e47b1de5a7bdbf8)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:30.655036 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65df0e3-a460-4e88-b113-44184c2625eb admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:30.792723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ccd44c3a1aa4edc88b261496cf58d6b', 'request_id': 'req-1046b9a3-452d-4822-8ab8-f994e9143c83'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:30.851573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b65df0e3-a460-4e88-b113-44184c2625eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ff84d4e534408e8e47b1de5a7bdbf8', 'request_id': 'req-b65df0e3-a460-4e88-b113-44184c2625eb'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:30.860490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b65df0e3-a460-4e88-b113-44184c2625eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ff84d4e534408e8e47b1de5a7bdbf8', 'request_id': 'req-b65df0e3-a460-4e88-b113-44184c2625eb'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:30.875966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b65df0e3-a460-4e88-b113-44184c2625eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ff84d4e534408e8e47b1de5a7bdbf8', 'request_id': 'req-b65df0e3-a460-4e88-b113-44184c2625eb'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:30.879128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ccd44c3a1aa4edc88b261496cf58d6b', 'request_id': 'req-1046b9a3-452d-4822-8ab8-f994e9143c83'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:30.890037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ccd44c3a1aa4edc88b261496cf58d6b', 'request_id': 'req-1046b9a3-452d-4822-8ab8-f994e9143c83'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:30.925880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] The token cache is being invalidate because project 5ccd44c3a1aa4edc88b261496cf58d6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:30.926608 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:30.928014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:30.949583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 952/1905] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:30 2022] DELETE /identity/v3/projects/5ccd44c3a1aa4edc88b261496cf58d6b => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:30.966360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:30.966360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:30.992053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b65df0e3-a460-4e88-b113-44184c2625eb admin admin] The token cache is being invalidate because project 81ff84d4e534408e8e47b1de5a7bdbf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:30.992422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b65df0e3-a460-4e88-b113-44184c2625eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:30.995769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b65df0e3-a460-4e88-b113-44184c2625eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:31.000301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 954/1906] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:30 2022] DELETE /identity/v3/projects/81ff84d4e534408e8e47b1de5a7bdbf8 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:31.005837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e9e42-2fe1-4baf-ad27-b0eee492c9c9 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:31.006128 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e9e42-2fe1-4baf-ad27-b0eee492c9c9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:31.006510 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e9e42-2fe1-4baf-ad27-b0eee492c9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:31.018777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:31.038428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-651e9e42-2fe1-4baf-ad27-b0eee492c9c9 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:31.039757 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-651e9e42-2fe1-4baf-ad27-b0eee492c9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:31.099058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046b9a3-452d-4822-8ab8-f994e9143c83 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:31.168730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 955/1907] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:31.179881 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3451d3-eca3-43e4-ab2b-fdd28f246de9 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:31.180413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3451d3-eca3-43e4-ab2b-fdd28f246de9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:31.180851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3451d3-eca3-43e4-ab2b-fdd28f246de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:31.201717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:31.203182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:31.203549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:31.203795 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] PATH_INFO: `/v3/users/910595392b9f4810af96fd73bc60314a` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:31.205144 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=910595392b9f4810af96fd73bc60314a)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:31.207251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:31.210618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ff3451d3-eca3-43e4-ab2b-fdd28f246de9 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:31.215893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ff3451d3-eca3-43e4-ab2b-fdd28f246de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:31.228878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 956/1908] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:31.234819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3451d3-eca3-43e4-ab2b-fdd28f246de9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:31.253050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3451d3-eca3-43e4-ab2b-fdd28f246de9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:31.264095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54928f22-0089-48e1-966d-225547f1a365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:31.264095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54928f22-0089-48e1-966d-225547f1a365 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:31.264095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54928f22-0089-48e1-966d-225547f1a365 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:31.264095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54928f22-0089-48e1-966d-225547f1a365 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:31.264679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54928f22-0089-48e1-966d-225547f1a365 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:05:31.264679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54928f22-0089-48e1-966d-225547f1a365 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:31.265182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54928f22-0089-48e1-966d-225547f1a365 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:31.292700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 957/1909] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:05:31.295024 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54928f22-0089-48e1-966d-225547f1a365 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:31.313146 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54928f22-0089-48e1-966d-225547f1a365 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:31.322076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6d64f-a97a-4238-9f0b-4afafd53bd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:31.323313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6d64f-a97a-4238-9f0b-4afafd53bd83 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:31.323834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6d64f-a97a-4238-9f0b-4afafd53bd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:31.324383 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6d64f-a97a-4238-9f0b-4afafd53bd83 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:31.327242 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6d64f-a97a-4238-9f0b-4afafd53bd83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:31.330419 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6d64f-a97a-4238-9f0b-4afafd53bd83 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:31.458789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 958/1910] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:31 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:31.463751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6d64f-a97a-4238-9f0b-4afafd53bd83 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:31.482373 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:31.504196 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910595392b9f4810af96fd73bc60314a'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:31.505231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910595392b9f4810af96fd73bc60314a'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:31.506396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910595392b9f4810af96fd73bc60314a'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:31.555296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6d64f-a97a-4238-9f0b-4afafd53bd83 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:31.560178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b86382-2371-4105-b8d5-56051138a05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:31.560935 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b86382-2371-4105-b8d5-56051138a05b admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:31.561356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b86382-2371-4105-b8d5-56051138a05b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:31.561787 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b86382-2371-4105-b8d5-56051138a05b admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:31.562806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b86382-2371-4105-b8d5-56051138a05b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:31.564744 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b86382-2371-4105-b8d5-56051138a05b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:31.627617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910595392b9f4810af96fd73bc60314a'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:31.641973 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910595392b9f4810af96fd73bc60314a'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:31.657802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 953/1911] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:30 2022] DELETE /identity/v3/users/910595392b9f4810af96fd73bc60314a => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:31.680947 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 959/1912] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:31 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:31.686182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:31.687962 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:31.875859 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695cdee-cbea-49c7-855e-a9d35e4e10b9 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:31.884929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e9a8f-843c-422d-8563-f26db810964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:31.885933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e9a8f-843c-422d-8563-f26db810964d admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:31.886418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e9a8f-843c-422d-8563-f26db810964d admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:31.886708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e9a8f-843c-422d-8563-f26db810964d admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:31.887464 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e9a8f-843c-422d-8563-f26db810964d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:31.889138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e9a8f-843c-422d-8563-f26db810964d admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:31.903532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 954/1913] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:31.908089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b86382-2371-4105-b8d5-56051138a05b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:31.919978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b86382-2371-4105-b8d5-56051138a05b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:31.933474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12673221-4f40-4f6a-8744-42018680a43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:31.934186 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12673221-4f40-4f6a-8744-42018680a43a admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:31.934531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12673221-4f40-4f6a-8744-42018680a43a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:31.934815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12673221-4f40-4f6a-8744-42018680a43a admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:31.935942 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12673221-4f40-4f6a-8744-42018680a43a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:31.941743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12673221-4f40-4f6a-8744-42018680a43a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:31.977692 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 960/1914] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:31.978142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e9a8f-843c-422d-8563-f26db810964d admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:31.997851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e9a8f-843c-422d-8563-f26db810964d admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:32.021683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d421199-7b4c-4289-bbb4-44e26f6cce3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:32.023292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d421199-7b4c-4289-bbb4-44e26f6cce3e admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:32.023736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d421199-7b4c-4289-bbb4-44e26f6cce3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:32.024162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d421199-7b4c-4289-bbb4-44e26f6cce3e admin admin] PATH_INFO: `/v3/projects/a10e6dddd9734560b2f20207cf874301/users/336cc460d29e4a27bc53066e386bd334/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:32.072432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d421199-7b4c-4289-bbb4-44e26f6cce3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a10e6dddd9734560b2f20207cf874301, user_id=336cc460d29e4a27bc53066e386bd334, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:32.076315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d421199-7b4c-4289-bbb4-44e26f6cce3e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:32.246202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4d876-bec3-4305-af63-402d21ef64ff None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:32.246895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4d876-bec3-4305-af63-402d21ef64ff None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:32.247382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4d876-bec3-4305-af63-402d21ef64ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:32.252162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 955/1915] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:31 2022] PUT /identity/v3/projects/a10e6dddd9734560b2f20207cf874301/users/336cc460d29e4a27bc53066e386bd334/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:32.286647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d4a4d876-bec3-4305-af63-402d21ef64ff None None] MFA Rules not processed for user `336cc460d29e4a27bc53066e386bd334`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:32.288480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d4a4d876-bec3-4305-af63-402d21ef64ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:32.362222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d421199-7b4c-4289-bbb4-44e26f6cce3e admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:32.366095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2d421199-7b4c-4289-bbb4-44e26f6cce3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:32.447716 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 961/1916] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:32.468748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4d876-bec3-4305-af63-402d21ef64ff None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:32.471002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d4a4d876-bec3-4305-af63-402d21ef64ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:32.605478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d421199-7b4c-4289-bbb4-44e26f6cce3e admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:32.616374 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:32.618276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:32.619072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:32.619775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] PATH_INFO: `/v3/projects/2e41c6e14ac5479094a374595ab6d93e` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:32.622592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e41c6e14ac5479094a374595ab6d93e)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:32.628562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:32.670286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4d876-bec3-4305-af63-402d21ef64ff None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:32.683463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611e2b0-b662-4025-b7fb-d22ef6851972 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:32.684151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611e2b0-b662-4025-b7fb-d22ef6851972 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:32.684449 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611e2b0-b662-4025-b7fb-d22ef6851972 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:32.684731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611e2b0-b662-4025-b7fb-d22ef6851972 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:32.702580 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:32.709493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611e2b0-b662-4025-b7fb-d22ef6851972 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:32.712821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611e2b0-b662-4025-b7fb-d22ef6851972 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:32.743515 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 962/1917] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:05:32.786030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611e2b0-b662-4025-b7fb-d22ef6851972 service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:32.825357 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611e2b0-b662-4025-b7fb-d22ef6851972 service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:32.849414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e41c6e14ac5479094a374595ab6d93e', 'request_id': 'req-4fa908aa-a053-4795-8bf4-d48e9049e6bf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:32.978124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e41c6e14ac5479094a374595ab6d93e', 'request_id': 'req-4fa908aa-a053-4795-8bf4-d48e9049e6bf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:32.983858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e41c6e14ac5479094a374595ab6d93e', 'request_id': 'req-4fa908aa-a053-4795-8bf4-d48e9049e6bf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:33.013348 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] The token cache is being invalidate because project 2e41c6e14ac5479094a374595ab6d93e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:33.013987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:33.015731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:33.019053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 956/1918] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:32 2022] DELETE /identity/v3/projects/2e41c6e14ac5479094a374595ab6d93e => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:33.028219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:33.032685 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:33.039384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d928b5-ee67-45e4-b588-0d2864090947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:33.040360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d928b5-ee67-45e4-b588-0d2864090947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:33.040660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d928b5-ee67-45e4-b588-0d2864090947 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:33.040943 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d928b5-ee67-45e4-b588-0d2864090947 admin admin] PATH_INFO: `/v3/users/336cc460d29e4a27bc53066e386bd334` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:33.043654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d928b5-ee67-45e4-b588-0d2864090947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=336cc460d29e4a27bc53066e386bd334)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:33.046112 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d928b5-ee67-45e4-b588-0d2864090947 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:33.073970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa908aa-a053-4795-8bf4-d48e9049e6bf admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:33.086419 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:33.088116 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:33.089153 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:33.090182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] PATH_INFO: `/v3/users/52fb0c35617147d89fc2b9564cddd7a4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:33.093544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=52fb0c35617147d89fc2b9564cddd7a4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:33.099617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:33.166614 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:33.391218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e4d928b5-ee67-45e4-b588-0d2864090947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336cc460d29e4a27bc53066e386bd334'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:33.392028 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e4d928b5-ee67-45e4-b588-0d2864090947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336cc460d29e4a27bc53066e386bd334'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:33.405821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e4d928b5-ee67-45e4-b588-0d2864090947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336cc460d29e4a27bc53066e386bd334'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:33.429662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e4d928b5-ee67-45e4-b588-0d2864090947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336cc460d29e4a27bc53066e386bd334'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:33.465228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52fb0c35617147d89fc2b9564cddd7a4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:33.465952 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52fb0c35617147d89fc2b9564cddd7a4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:33.466722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52fb0c35617147d89fc2b9564cddd7a4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:33.540730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e4d928b5-ee67-45e4-b588-0d2864090947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336cc460d29e4a27bc53066e386bd334'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:33.554404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 963/1919] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:32 2022] DELETE /identity/v3/users/336cc460d29e4a27bc53066e386bd334 => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:33.561773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb17e2-15fc-4425-bcc8-d7dcb5e10ae9 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:33.562299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb17e2-15fc-4425-bcc8-d7dcb5e10ae9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:33.563974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb17e2-15fc-4425-bcc8-d7dcb5e10ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:33.589095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-87bb17e2-15fc-4425-bcc8-d7dcb5e10ae9 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:33.590988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-87bb17e2-15fc-4425-bcc8-d7dcb5e10ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:33.662976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52fb0c35617147d89fc2b9564cddd7a4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:33.685996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52fb0c35617147d89fc2b9564cddd7a4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:33.710671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 964/1920] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:33.719714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 957/1921] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:33 2022] DELETE /identity/v3/users/52fb0c35617147d89fc2b9564cddd7a4 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:33.723205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb17e2-15fc-4425-bcc8-d7dcb5e10ae9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:33.727292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-87bb17e2-15fc-4425-bcc8-d7dcb5e10ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:33.830461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb17e2-15fc-4425-bcc8-d7dcb5e10ae9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:33.911025 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02138dd-a7b4-4b81-8914-eee8492b3c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:33.912018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02138dd-a7b4-4b81-8914-eee8492b3c4f service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:33.912649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02138dd-a7b4-4b81-8914-eee8492b3c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:33.913030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02138dd-a7b4-4b81-8914-eee8492b3c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:34.096331 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02138dd-a7b4-4b81-8914-eee8492b3c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:34.097753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02138dd-a7b4-4b81-8914-eee8492b3c4f service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:34.113464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 965/1922] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:34.457627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:34.470536 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:34.473941 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:34.511846 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8ea31-b1be-4c44-be77-b9a11ddddc6a admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:34.522523 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:34.523572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:34.523867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:34.524154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e admin admin] PATH_INFO: `/v3/projects/eff99571dd3849168370f92b2690bdee` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:34.525623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff99571dd3849168370f92b2690bdee)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:34.527685 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:34.738407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff99571dd3849168370f92b2690bdee', 'request_id': 'req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:34.799747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02138dd-a7b4-4b81-8914-eee8492b3c4f service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:34.804585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f02138dd-a7b4-4b81-8914-eee8492b3c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:34.816575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02138dd-a7b4-4b81-8914-eee8492b3c4f service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:34.826051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff99571dd3849168370f92b2690bdee', 'request_id': 'req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:34.843721 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff99571dd3849168370f92b2690bdee', 'request_id': 'req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:34.869324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e admin admin] The token cache is being invalidate because project eff99571dd3849168370f92b2690bdee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:34.869533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:34.870377 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-5ae6ac61-b1f3-494c-8ba7-1b1a14e6ad3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:34.872878 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 958/1923] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:34 2022] DELETE /identity/v3/projects/eff99571dd3849168370f92b2690bdee => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:34.880458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e524a-bcb2-4472-8532-eb199a62a3fe None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:34.880981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e524a-bcb2-4472-8532-eb199a62a3fe None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:34.881375 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e524a-bcb2-4472-8532-eb199a62a3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:34.899983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008cdb4-c22d-4e51-9dac-b0559b813d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:34.900976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008cdb4-c22d-4e51-9dac-b0559b813d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:34.901320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008cdb4-c22d-4e51-9dac-b0559b813d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:34.901589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008cdb4-c22d-4e51-9dac-b0559b813d33 admin admin] PATH_INFO: `/v3/projects/a10e6dddd9734560b2f20207cf874301` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:34.903032 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008cdb4-c22d-4e51-9dac-b0559b813d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a10e6dddd9734560b2f20207cf874301)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:34.906261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008cdb4-c22d-4e51-9dac-b0559b813d33 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:34.913041 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-cf3e524a-bcb2-4472-8532-eb199a62a3fe None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:34.914457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-cf3e524a-bcb2-4472-8532-eb199a62a3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:34.939847 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 959/1924] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:34.939847 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90223986-290b-4efb-8cde-750324c362b2 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:34.940124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90223986-290b-4efb-8cde-750324c362b2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:34.940312 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90223986-290b-4efb-8cde-750324c362b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:34.963243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-90223986-290b-4efb-8cde-750324c362b2 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:34.964487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-90223986-290b-4efb-8cde-750324c362b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:34.974879 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 960/1925] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:34.979166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90223986-290b-4efb-8cde-750324c362b2 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:34.990353 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90223986-290b-4efb-8cde-750324c362b2 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:34.995784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9276f4-3ede-419a-be6a-5fa999ffc2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:35.000787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9276f4-3ede-419a-be6a-5fa999ffc2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.000787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9276f4-3ede-419a-be6a-5fa999ffc2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.000787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9276f4-3ede-419a-be6a-5fa999ffc2b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.000787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9276f4-3ede-419a-be6a-5fa999ffc2b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:05:35.000787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9276f4-3ede-419a-be6a-5fa999ffc2b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:35.000787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9276f4-3ede-419a-be6a-5fa999ffc2b9 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:35.015157 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 961/1926] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:05:35.020063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9276f4-3ede-419a-be6a-5fa999ffc2b9 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:35.031340 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9276f4-3ede-419a-be6a-5fa999ffc2b9 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:35.043102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc174984-5223-4176-b19b-7ce4cdb47900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:35.044251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc174984-5223-4176-b19b-7ce4cdb47900 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.044775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc174984-5223-4176-b19b-7ce4cdb47900 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.047179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc174984-5223-4176-b19b-7ce4cdb47900 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.047179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc174984-5223-4176-b19b-7ce4cdb47900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:35.049602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc174984-5223-4176-b19b-7ce4cdb47900 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:35.140055 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7008cdb4-c22d-4e51-9dac-b0559b813d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a10e6dddd9734560b2f20207cf874301', 'request_id': 'req-7008cdb4-c22d-4e51-9dac-b0559b813d33'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:35.147392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7008cdb4-c22d-4e51-9dac-b0559b813d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a10e6dddd9734560b2f20207cf874301', 'request_id': 'req-7008cdb4-c22d-4e51-9dac-b0559b813d33'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:35.154365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7008cdb4-c22d-4e51-9dac-b0559b813d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a10e6dddd9734560b2f20207cf874301', 'request_id': 'req-7008cdb4-c22d-4e51-9dac-b0559b813d33'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:35.200666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 962/1927] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:35 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:05:35.211110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc174984-5223-4176-b19b-7ce4cdb47900 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:35.268840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7008cdb4-c22d-4e51-9dac-b0559b813d33 admin admin] The token cache is being invalidate because project a10e6dddd9734560b2f20207cf874301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:35.269437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7008cdb4-c22d-4e51-9dac-b0559b813d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:35.270369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7008cdb4-c22d-4e51-9dac-b0559b813d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:35.273051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 966/1928] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:34 2022] DELETE /identity/v3/projects/a10e6dddd9734560b2f20207cf874301 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:35.278049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f4bcd-50c3-47dc-8465-d57b64b2a8ce None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.278506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f4bcd-50c3-47dc-8465-d57b64b2a8ce None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.279003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f4bcd-50c3-47dc-8465-d57b64b2a8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.303000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc174984-5223-4176-b19b-7ce4cdb47900 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:35.304517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-451f4bcd-50c3-47dc-8465-d57b64b2a8ce None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:35.305831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-451f4bcd-50c3-47dc-8465-d57b64b2a8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:35.308599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acca92-2c02-4825-aa9e-6e7b758a50e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:35.309437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acca92-2c02-4825-aa9e-6e7b758a50e0 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.309889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acca92-2c02-4825-aa9e-6e7b758a50e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.310841 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acca92-2c02-4825-aa9e-6e7b758a50e0 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.311601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7acca92-2c02-4825-aa9e-6e7b758a50e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:35.314015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7acca92-2c02-4825-aa9e-6e7b758a50e0 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:35.315980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 967/1929] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:35.332704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe04c5-3ecb-4c91-991e-9d5ded811154 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.333568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe04c5-3ecb-4c91-991e-9d5ded811154 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.333761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe04c5-3ecb-4c91-991e-9d5ded811154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.362751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-46fe04c5-3ecb-4c91-991e-9d5ded811154 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:35.364070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-46fe04c5-3ecb-4c91-991e-9d5ded811154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:35.373558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 968/1930] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:35.378064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe04c5-3ecb-4c91-991e-9d5ded811154 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:35.388785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe04c5-3ecb-4c91-991e-9d5ded811154 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:35.414108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7396ed6-cdfc-4b41-80a6-479dc4f46665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:35.414108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7396ed6-cdfc-4b41-80a6-479dc4f46665 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.414108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7396ed6-cdfc-4b41-80a6-479dc4f46665 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.414108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7396ed6-cdfc-4b41-80a6-479dc4f46665 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.414906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7396ed6-cdfc-4b41-80a6-479dc4f46665 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:05:35.414906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7396ed6-cdfc-4b41-80a6-479dc4f46665 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:35.414906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7396ed6-cdfc-4b41-80a6-479dc4f46665 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:35.422114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 969/1931] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:05:35.427417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7396ed6-cdfc-4b41-80a6-479dc4f46665 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:35.444093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7396ed6-cdfc-4b41-80a6-479dc4f46665 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:35.449680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4b0c3-0c70-4b9b-874b-6b70605bc3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:35.450722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4b0c3-0c70-4b9b-874b-6b70605bc3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.451091 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4b0c3-0c70-4b9b-874b-6b70605bc3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.455213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4b0c3-0c70-4b9b-874b-6b70605bc3a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.455213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4b0c3-0c70-4b9b-874b-6b70605bc3a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:35.455213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4b0c3-0c70-4b9b-874b-6b70605bc3a0 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:35.526375 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 963/1932] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:35 2022] POST /identity/v3/users => generated 289 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:05:35.531614 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acca92-2c02-4825-aa9e-6e7b758a50e0 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:35.533480 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f7acca92-2c02-4825-aa9e-6e7b758a50e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:35.548551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acca92-2c02-4825-aa9e-6e7b758a50e0 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:35.554681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f020378-caac-4848-b8b9-c2fd26e98d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:35.556758 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f020378-caac-4848-b8b9-c2fd26e98d4a admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.556758 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f020378-caac-4848-b8b9-c2fd26e98d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.556758 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f020378-caac-4848-b8b9-c2fd26e98d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.556758 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f020378-caac-4848-b8b9-c2fd26e98d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:35.559312 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f020378-caac-4848-b8b9-c2fd26e98d4a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:35.571568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 964/1933] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:35.575551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f020378-caac-4848-b8b9-c2fd26e98d4a admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:35.587518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f020378-caac-4848-b8b9-c2fd26e98d4a admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:35.594179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 970/1934] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:35 2022] POST /identity/v3/projects => generated 367 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:05:35.595745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4b0c3-0c70-4b9b-874b-6b70605bc3a0 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:35.658522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=75,keystone:DELETE=22,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:35.666175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c176566-eb16-41ec-91d5-72cd16ce6c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:35.666490 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c176566-eb16-41ec-91d5-72cd16ce6c22 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.666969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c176566-eb16-41ec-91d5-72cd16ce6c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.667427 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c176566-eb16-41ec-91d5-72cd16ce6c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.668557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c176566-eb16-41ec-91d5-72cd16ce6c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:35.670874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c176566-eb16-41ec-91d5-72cd16ce6c22 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:35.688999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 965/1935] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:35.690347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c176566-eb16-41ec-91d5-72cd16ce6c22 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:35.691836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4b0c3-0c70-4b9b-874b-6b70605bc3a0 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:35.702887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73917d-8660-4da8-a173-cf89f4d4c284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:35.702887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73917d-8660-4da8-a173-cf89f4d4c284 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.702887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73917d-8660-4da8-a173-cf89f4d4c284 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.702887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73917d-8660-4da8-a173-cf89f4d4c284 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.703229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73917d-8660-4da8-a173-cf89f4d4c284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:35.703481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c176566-eb16-41ec-91d5-72cd16ce6c22 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:35.705537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73917d-8660-4da8-a173-cf89f4d4c284 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:35.709622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53720640-0893-4735-8b41-6d63d0029193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:35.711373 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720640-0893-4735-8b41-6d63d0029193 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.711877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720640-0893-4735-8b41-6d63d0029193 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.712325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720640-0893-4735-8b41-6d63d0029193 admin admin] PATH_INFO: `/v3/projects/2f068693208345f88cf658bc625fb4e1/users/a878b057245340e6990c39ccddfd4bf4/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.731149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53720640-0893-4735-8b41-6d63d0029193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f068693208345f88cf658bc625fb4e1, user_id=a878b057245340e6990c39ccddfd4bf4, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:35.734396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53720640-0893-4735-8b41-6d63d0029193 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:35.791612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 971/1936] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:35 2022] POST /identity/v3/users => generated 284 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:35.795968 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73917d-8660-4da8-a173-cf89f4d4c284 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:35.806972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73917d-8660-4da8-a173-cf89f4d4c284 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:35.813218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc37223-9746-4fd6-b877-e78b87cca2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:35.813624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc37223-9746-4fd6-b877-e78b87cca2ca admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.813912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc37223-9746-4fd6-b877-e78b87cca2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.814191 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc37223-9746-4fd6-b877-e78b87cca2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.815002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc37223-9746-4fd6-b877-e78b87cca2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:35.816668 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc37223-9746-4fd6-b877-e78b87cca2ca admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:35.826050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 966/1937] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:35 2022] PUT /identity/v3/projects/2f068693208345f88cf658bc625fb4e1/users/a878b057245340e6990c39ccddfd4bf4/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:35.829380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 972/1938] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:35.837360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53720640-0893-4735-8b41-6d63d0029193 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:35.848209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a098d4f-f8d4-4348-886c-992cf33a3ef1 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.848668 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a098d4f-f8d4-4348-886c-992cf33a3ef1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.849014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a098d4f-f8d4-4348-886c-992cf33a3ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.883657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-7a098d4f-f8d4-4348-886c-992cf33a3ef1 None None] MFA Rules not processed for user `a878b057245340e6990c39ccddfd4bf4`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:35.885218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7a098d4f-f8d4-4348-886c-992cf33a3ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:35.922170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53720640-0893-4735-8b41-6d63d0029193 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:35.927503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f325371-3581-4b0e-9a92-75f9f4b737c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:35.928192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f325371-3581-4b0e-9a92-75f9f4b737c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.928479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f325371-3581-4b0e-9a92-75f9f4b737c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.928912 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f325371-3581-4b0e-9a92-75f9f4b737c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.929664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f325371-3581-4b0e-9a92-75f9f4b737c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:35.931345 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f325371-3581-4b0e-9a92-75f9f4b737c1 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:35.947056 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 967/1939] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:35.949300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f325371-3581-4b0e-9a92-75f9f4b737c1 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:35.962664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f325371-3581-4b0e-9a92-75f9f4b737c1 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:35.967227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 973/1940] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:35.968868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4fde9-ae05-4787-95e5-c44d237e527c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:35.969798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4fde9-ae05-4787-95e5-c44d237e527c admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.970092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4fde9-ae05-4787-95e5-c44d237e527c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.970408 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4fde9-ae05-4787-95e5-c44d237e527c admin admin] PATH_INFO: `/v3/projects/9062c0ee33bb464c88d5f5e7471c6961/users/8d04f1378edc435a801a6c2719ec31f5/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.972007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a098d4f-f8d4-4348-886c-992cf33a3ef1 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:35.984588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a098d4f-f8d4-4348-886c-992cf33a3ef1 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:35.987411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4fde9-ae05-4787-95e5-c44d237e527c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9062c0ee33bb464c88d5f5e7471c6961, user_id=8d04f1378edc435a801a6c2719ec31f5, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:35.989431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82410b-d0b5-430e-ad8d-d44709199624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:35.990237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4fde9-ae05-4787-95e5-c44d237e527c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:35.990398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82410b-d0b5-430e-ad8d-d44709199624 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:35.990599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82410b-d0b5-430e-ad8d-d44709199624 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:35.990882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82410b-d0b5-430e-ad8d-d44709199624 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:35.991599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a82410b-d0b5-430e-ad8d-d44709199624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:35.993577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a82410b-d0b5-430e-ad8d-d44709199624 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:36.090223 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 968/1941] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:35 2022] PUT /identity/v3/projects/9062c0ee33bb464c88d5f5e7471c6961/users/8d04f1378edc435a801a6c2719ec31f5/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:36.096614 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7f2b6-f38f-437a-8916-a9ed8c331198 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:36.096909 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7f2b6-f38f-437a-8916-a9ed8c331198 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:36.097200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7f2b6-f38f-437a-8916-a9ed8c331198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:36.125787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-49d7f2b6-f38f-437a-8916-a9ed8c331198 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:36.128051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-49d7f2b6-f38f-437a-8916-a9ed8c331198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:36.133755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 974/1942] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:35 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:36.135516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82410b-d0b5-430e-ad8d-d44709199624 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:36.243822 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82410b-d0b5-430e-ad8d-d44709199624 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:36.248319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 969/1943] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:36.252806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba55aa-3f97-4733-86dc-97772900850e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:36.253752 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba55aa-3f97-4733-86dc-97772900850e admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:36.254160 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba55aa-3f97-4733-86dc-97772900850e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:36.254541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba55aa-3f97-4733-86dc-97772900850e admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:36.255357 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba55aa-3f97-4733-86dc-97772900850e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:36.258163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba55aa-3f97-4733-86dc-97772900850e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:36.259684 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7f2b6-f38f-437a-8916-a9ed8c331198 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:36.261303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-49d7f2b6-f38f-437a-8916-a9ed8c331198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:36.328655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7f2b6-f38f-437a-8916-a9ed8c331198 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:36.333035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e7699-0a92-4bbb-8afb-f19cdd134752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:36.333721 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e7699-0a92-4bbb-8afb-f19cdd134752 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:36.334006 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e7699-0a92-4bbb-8afb-f19cdd134752 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:36.334293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e7699-0a92-4bbb-8afb-f19cdd134752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:36.346605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e7699-0a92-4bbb-8afb-f19cdd134752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:36.348139 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e7699-0a92-4bbb-8afb-f19cdd134752 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:36.357625 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 975/1944] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:36 2022] POST /identity/v3/users => generated 288 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:36.359491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 970/1945] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:36.363581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba55aa-3f97-4733-86dc-97772900850e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:36.369369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:36.375222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba55aa-3f97-4733-86dc-97772900850e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:36.380844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77907b1d-bc17-4554-a7c8-0d6366598d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:36.381601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77907b1d-bc17-4554-a7c8-0d6366598d37 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:36.381890 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77907b1d-bc17-4554-a7c8-0d6366598d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:36.382175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77907b1d-bc17-4554-a7c8-0d6366598d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:36.382986 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77907b1d-bc17-4554-a7c8-0d6366598d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:36.385050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77907b1d-bc17-4554-a7c8-0d6366598d37 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:36.397196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 976/1946] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:36.401796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e7699-0a92-4bbb-8afb-f19cdd134752 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:36.413270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e7699-0a92-4bbb-8afb-f19cdd134752 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:36.419319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31040a0d-fab8-4c74-addb-abb605b4f989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:36.420279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31040a0d-fab8-4c74-addb-abb605b4f989 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:36.420675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31040a0d-fab8-4c74-addb-abb605b4f989 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:36.421075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31040a0d-fab8-4c74-addb-abb605b4f989 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:36.422229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31040a0d-fab8-4c74-addb-abb605b4f989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:36.424182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31040a0d-fab8-4c74-addb-abb605b4f989 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:36.437530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 971/1947] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:36.442312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77907b1d-bc17-4554-a7c8-0d6366598d37 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:36.454534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77907b1d-bc17-4554-a7c8-0d6366598d37 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:36.462115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545761b-10d3-4d43-bc56-3448581f8a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:36.462994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545761b-10d3-4d43-bc56-3448581f8a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:36.463273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545761b-10d3-4d43-bc56-3448581f8a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:36.463563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545761b-10d3-4d43-bc56-3448581f8a30 admin admin] PATH_INFO: `/v3/projects/97c244082b9d4c3dbbc12b16abb6c971/users/2803acbf41174a5ab942538477236ae8/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:36.485071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545761b-10d3-4d43-bc56-3448581f8a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c244082b9d4c3dbbc12b16abb6c971, user_id=2803acbf41174a5ab942538477236ae8, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:36.488976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545761b-10d3-4d43-bc56-3448581f8a30 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:36.596605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 977/1948] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:36 2022] PUT /identity/v3/projects/97c244082b9d4c3dbbc12b16abb6c971/users/2803acbf41174a5ab942538477236ae8/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:36.615269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4851cf-2165-4de9-85c5-13f80347c6e8 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:36.615994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4851cf-2165-4de9-85c5-13f80347c6e8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:36.616576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4851cf-2165-4de9-85c5-13f80347c6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:36.648246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3b4851cf-2165-4de9-85c5-13f80347c6e8 None None] MFA Rules not processed for user `2803acbf41174a5ab942538477236ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:36.650066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3b4851cf-2165-4de9-85c5-13f80347c6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:36.723157 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 972/1949] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:36.728113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545761b-10d3-4d43-bc56-3448581f8a30 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:36.843521 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545761b-10d3-4d43-bc56-3448581f8a30 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:36.854741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dac442-91d9-4427-b4ec-0460a41930c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:36.856519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dac442-91d9-4427-b4ec-0460a41930c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:36.857370 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dac442-91d9-4427-b4ec-0460a41930c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:36.858614 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dac442-91d9-4427-b4ec-0460a41930c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:36.860068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dac442-91d9-4427-b4ec-0460a41930c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:36.864311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dac442-91d9-4427-b4ec-0460a41930c2 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:36.887758 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:36.988929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 978/1950] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:36 2022] POST /identity/v3/projects => generated 375 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:36.992188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4851cf-2165-4de9-85c5-13f80347c6e8 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:37.004204 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4851cf-2165-4de9-85c5-13f80347c6e8 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:37.077245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc093f83-45e4-4064-8161-c741601a0670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:37.078190 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc093f83-45e4-4064-8161-c741601a0670 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:37.078671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc093f83-45e4-4064-8161-c741601a0670 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:37.078964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc093f83-45e4-4064-8161-c741601a0670 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:37.079684 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc093f83-45e4-4064-8161-c741601a0670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:37.081712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc093f83-45e4-4064-8161-c741601a0670 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:37.141500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:37.195918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 973/1951] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:36 2022] POST /identity/v3/users => generated 294 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:05:37.199824 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dac442-91d9-4427-b4ec-0460a41930c2 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:37.221922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dac442-91d9-4427-b4ec-0460a41930c2 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:37.230982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa8603-fd15-4c51-9870-19a012f39caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:37.232104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa8603-fd15-4c51-9870-19a012f39caf admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:37.232504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa8603-fd15-4c51-9870-19a012f39caf admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:37.232926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa8603-fd15-4c51-9870-19a012f39caf admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:37.233938 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa8603-fd15-4c51-9870-19a012f39caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:37.237232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa8603-fd15-4c51-9870-19a012f39caf admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:37.258268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 979/1952] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:37.260330 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc093f83-45e4-4064-8161-c741601a0670 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:37.275230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc093f83-45e4-4064-8161-c741601a0670 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:37.295819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c9036-fd1b-4db9-8673-04a7ec27fc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:37.296769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c9036-fd1b-4db9-8673-04a7ec27fc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:37.297057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c9036-fd1b-4db9-8673-04a7ec27fc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:37.297397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c9036-fd1b-4db9-8673-04a7ec27fc69 admin admin] PATH_INFO: `/v3/projects/3f8287dbf5554b428dd8fb8a766601aa/users/a0f35828d260447593c7d3602a1c2cf7/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:37.315342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c9036-fd1b-4db9-8673-04a7ec27fc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f8287dbf5554b428dd8fb8a766601aa, user_id=a0f35828d260447593c7d3602a1c2cf7, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:37.318065 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c9036-fd1b-4db9-8673-04a7ec27fc69 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:37.439648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 974/1953] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:37 2022] PUT /identity/v3/projects/3f8287dbf5554b428dd8fb8a766601aa/users/a0f35828d260447593c7d3602a1c2cf7/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:37.443978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa8603-fd15-4c51-9870-19a012f39caf admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:37.457161 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa8603-fd15-4c51-9870-19a012f39caf admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:37.522581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc3963-7420-4846-b800-f23bbde8b673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:37.523194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc3963-7420-4846-b800-f23bbde8b673 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:37.523487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc3963-7420-4846-b800-f23bbde8b673 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:37.523794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc3963-7420-4846-b800-f23bbde8b673 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:37.524534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc3963-7420-4846-b800-f23bbde8b673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:37.526167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc3963-7420-4846-b800-f23bbde8b673 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:37.537213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 980/1954] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:37.543898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c9036-fd1b-4db9-8673-04a7ec27fc69 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:37.561999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c9036-fd1b-4db9-8673-04a7ec27fc69 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:37.570169 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053162da-5f05-4ea8-8a2c-33d38ba4fa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:37.571710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053162da-5f05-4ea8-8a2c-33d38ba4fa99 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:37.572093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053162da-5f05-4ea8-8a2c-33d38ba4fa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:37.572572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053162da-5f05-4ea8-8a2c-33d38ba4fa99 admin admin] PATH_INFO: `/v3/projects/3f8287dbf5554b428dd8fb8a766601aa/users/a0f35828d260447593c7d3602a1c2cf7/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:37.575929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053162da-5f05-4ea8-8a2c-33d38ba4fa99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f8287dbf5554b428dd8fb8a766601aa, user_id=a0f35828d260447593c7d3602a1c2cf7, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:37.581695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053162da-5f05-4ea8-8a2c-33d38ba4fa99 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:37.600670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 975/1955] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:37 2022] PUT /identity/v3/projects/3f8287dbf5554b428dd8fb8a766601aa/users/a0f35828d260447593c7d3602a1c2cf7/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:37.618976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9f9f0-26d5-4a85-a829-1ed7169b82cf None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:37.619273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9f9f0-26d5-4a85-a829-1ed7169b82cf None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:37.619564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9f9f0-26d5-4a85-a829-1ed7169b82cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:37.652221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-e9a9f9f0-26d5-4a85-a829-1ed7169b82cf None None] MFA Rules not processed for user `a0f35828d260447593c7d3602a1c2cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:37.652847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e9a9f9f0-26d5-4a85-a829-1ed7169b82cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:37.733738 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 981/1956] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:37 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:37.745901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:37.747835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053162da-5f05-4ea8-8a2c-33d38ba4fa99 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:37.750620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-053162da-5f05-4ea8-8a2c-33d38ba4fa99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:37.822688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053162da-5f05-4ea8-8a2c-33d38ba4fa99 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:37.828956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb88b8-b6ac-42ae-aaf9-3cb93267a06a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:37.829646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb88b8-b6ac-42ae-aaf9-3cb93267a06a service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:37.829937 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb88b8-b6ac-42ae-aaf9-3cb93267a06a service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:37.830220 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb88b8-b6ac-42ae-aaf9-3cb93267a06a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:37.906041 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb88b8-b6ac-42ae-aaf9-3cb93267a06a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:37.908464 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb88b8-b6ac-42ae-aaf9-3cb93267a06a service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:37.923591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 976/1957] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:38.136785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:38.251022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:38.264214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346d61f-f1e1-4c46-976b-f4719aaea13e None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:38.265532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346d61f-f1e1-4c46-976b-f4719aaea13e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:38.266451 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346d61f-f1e1-4c46-976b-f4719aaea13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:38.334199 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-7346d61f-f1e1-4c46-976b-f4719aaea13e None None] MFA Rules not processed for user `8d04f1378edc435a801a6c2719ec31f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:38.335584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7346d61f-f1e1-4c46-976b-f4719aaea13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:38.410923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 982/1958] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:38.426867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb88b8-b6ac-42ae-aaf9-3cb93267a06a service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:38.426867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-26fb88b8-b6ac-42ae-aaf9-3cb93267a06a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:38.440077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb88b8-b6ac-42ae-aaf9-3cb93267a06a service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:38.445652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cfd9e-0ce3-43e7-824f-50871ea250f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:38.446261 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cfd9e-0ce3-43e7-824f-50871ea250f7 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:38.446598 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cfd9e-0ce3-43e7-824f-50871ea250f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:38.446868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cfd9e-0ce3-43e7-824f-50871ea250f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:38.458217 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cfd9e-0ce3-43e7-824f-50871ea250f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:38.459720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cfd9e-0ce3-43e7-824f-50871ea250f7 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:38.471406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 977/1959] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:39.111469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:41.220402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346d61f-f1e1-4c46-976b-f4719aaea13e None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:41.230515 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346d61f-f1e1-4c46-976b-f4719aaea13e None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:41.235109 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bd754-aa79-40f8-b111-352e6d32363c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:41.235790 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bd754-aa79-40f8-b111-352e6d32363c service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:41.236078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bd754-aa79-40f8-b111-352e6d32363c service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:41.236362 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bd754-aa79-40f8-b111-352e6d32363c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:41.302389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bd754-aa79-40f8-b111-352e6d32363c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:41.303923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bd754-aa79-40f8-b111-352e6d32363c service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:41.314066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 983/1960] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:05:41.860122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73286660-cff8-429c-83b6-1566431cfd5b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:41.860784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73286660-cff8-429c-83b6-1566431cfd5b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:41.860784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73286660-cff8-429c-83b6-1566431cfd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:41.893034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-73286660-cff8-429c-83b6-1566431cfd5b None None] MFA Rules not processed for user `8d04f1378edc435a801a6c2719ec31f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:41.894772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-73286660-cff8-429c-83b6-1566431cfd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:41.904182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 978/1961] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:42.938090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bd754-aa79-40f8-b111-352e6d32363c service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:42.942981 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-bc2bd754-aa79-40f8-b111-352e6d32363c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:43.040504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bd754-aa79-40f8-b111-352e6d32363c service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:43.046085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c9d72-f878-4e6b-be3d-bd8af5fd54b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:43.047021 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c9d72-f878-4e6b-be3d-bd8af5fd54b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:43.047308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c9d72-f878-4e6b-be3d-bd8af5fd54b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:43.047590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c9d72-f878-4e6b-be3d-bd8af5fd54b2 admin admin] PATH_INFO: `/v3/users/2a045745710442b098058c68c3689711` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:43.048926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c9d72-f878-4e6b-be3d-bd8af5fd54b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a045745710442b098058c68c3689711)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:43.051179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c9d72-f878-4e6b-be3d-bd8af5fd54b2 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:43.058679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:43.307215 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-608c9d72-f878-4e6b-be3d-bd8af5fd54b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a045745710442b098058c68c3689711'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:43.308296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-608c9d72-f878-4e6b-be3d-bd8af5fd54b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a045745710442b098058c68c3689711'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:43.349568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-608c9d72-f878-4e6b-be3d-bd8af5fd54b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a045745710442b098058c68c3689711'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:43.370555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-608c9d72-f878-4e6b-be3d-bd8af5fd54b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a045745710442b098058c68c3689711'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:43.456604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-608c9d72-f878-4e6b-be3d-bd8af5fd54b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a045745710442b098058c68c3689711'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:43.462587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73286660-cff8-429c-83b6-1566431cfd5b None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:43.469487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73286660-cff8-429c-83b6-1566431cfd5b None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:43.471572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 984/1962] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:42 2022] DELETE /identity/v3/users/2a045745710442b098058c68c3689711 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:43.474620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45e789-7eec-4c23-aa87-d74468478cbf None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:43.474620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45e789-7eec-4c23-aa87-d74468478cbf None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:43.474620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45e789-7eec-4c23-aa87-d74468478cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:43.541051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ef45e789-7eec-4c23-aa87-d74468478cbf None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:43.542911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ef45e789-7eec-4c23-aa87-d74468478cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:43.577281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e98480-bad3-43d1-a08f-0e290e2965f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:43.577281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98480-bad3-43d1-a08f-0e290e2965f6 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:43.577281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98480-bad3-43d1-a08f-0e290e2965f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:43.577281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98480-bad3-43d1-a08f-0e290e2965f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:43.579501 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e3e98480-bad3-43d1-a08f-0e290e2965f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:43.647318 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 985/1963] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:43.657395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45e789-7eec-4c23-aa87-d74468478cbf None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:43.659149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ef45e789-7eec-4c23-aa87-d74468478cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:43.671106 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e98480-bad3-43d1-a08f-0e290e2965f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:43.672979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e98480-bad3-43d1-a08f-0e290e2965f6 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:43.686559 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 979/1964] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:05:43.741284 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45e789-7eec-4c23-aa87-d74468478cbf None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:43.745774 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42f865-0081-4e7c-a08b-3154f738d18a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:43.746737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42f865-0081-4e7c-a08b-3154f738d18a service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:43.747097 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42f865-0081-4e7c-a08b-3154f738d18a service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:43.747385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42f865-0081-4e7c-a08b-3154f738d18a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:43.761008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42f865-0081-4e7c-a08b-3154f738d18a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:43.763630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42f865-0081-4e7c-a08b-3154f738d18a service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:43.775091 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 986/1965] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:05:44.439908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e98480-bad3-43d1-a08f-0e290e2965f6 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:44.441863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e3e98480-bad3-43d1-a08f-0e290e2965f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:44.458928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e98480-bad3-43d1-a08f-0e290e2965f6 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:44.464668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:44.465647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:44.465827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:44.466364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] PATH_INFO: `/v3/projects/808e941babe345a1bed7bcd020f9652d` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:44.468455 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] RBAC: Authorizing `identity:delete_project(project_id=808e941babe345a1bed7bcd020f9652d)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:44.471856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:44.655951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:44.662699 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '808e941babe345a1bed7bcd020f9652d', 'request_id': 'req-f5d31764-ab2a-4213-b57f-76403e69d9af'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:44.785677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '808e941babe345a1bed7bcd020f9652d', 'request_id': 'req-f5d31764-ab2a-4213-b57f-76403e69d9af'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:44.792681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '808e941babe345a1bed7bcd020f9652d', 'request_id': 'req-f5d31764-ab2a-4213-b57f-76403e69d9af'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:44.808658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] The token cache is being invalidate because project 808e941babe345a1bed7bcd020f9652d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:44.808658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:44.809406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:44.812162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 980/1966] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:44 2022] DELETE /identity/v3/projects/808e941babe345a1bed7bcd020f9652d => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:44.822791 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42f865-0081-4e7c-a08b-3154f738d18a service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:44.833886 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42f865-0081-4e7c-a08b-3154f738d18a service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:44.864983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:44.899275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e9ffc-456f-4554-95d1-0eea4a97b1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:44.900330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e9ffc-456f-4554-95d1-0eea4a97b1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:44.900695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e9ffc-456f-4554-95d1-0eea4a97b1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:44.900894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e9ffc-456f-4554-95d1-0eea4a97b1e3 admin admin] PATH_INFO: `/v3/users/064180adf52d40b2bd5087cce1fcbdec` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:44.902214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e9ffc-456f-4554-95d1-0eea4a97b1e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=064180adf52d40b2bd5087cce1fcbdec)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:44.904295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e9ffc-456f-4554-95d1-0eea4a97b1e3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:45.294912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-047e9ffc-456f-4554-95d1-0eea4a97b1e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '064180adf52d40b2bd5087cce1fcbdec'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:45.296169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-047e9ffc-456f-4554-95d1-0eea4a97b1e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '064180adf52d40b2bd5087cce1fcbdec'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:45.305071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-047e9ffc-456f-4554-95d1-0eea4a97b1e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '064180adf52d40b2bd5087cce1fcbdec'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:45.319184 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-047e9ffc-456f-4554-95d1-0eea4a97b1e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '064180adf52d40b2bd5087cce1fcbdec'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:45.414271 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-047e9ffc-456f-4554-95d1-0eea4a97b1e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '064180adf52d40b2bd5087cce1fcbdec'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:45.423679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 987/1967] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:44 2022] DELETE /identity/v3/users/064180adf52d40b2bd5087cce1fcbdec => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:46.057713 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:46.059084 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:46.090982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d31764-ab2a-4213-b57f-76403e69d9af admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:46.181211 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74133ae1-5e91-46e0-8220-083250c1db16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:46.182156 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74133ae1-5e91-46e0-8220-083250c1db16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:46.182490 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74133ae1-5e91-46e0-8220-083250c1db16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:46.182777 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74133ae1-5e91-46e0-8220-083250c1db16 admin admin] PATH_INFO: `/v3/projects/21053db796e4440bb8901bca240edacd` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:46.184324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74133ae1-5e91-46e0-8220-083250c1db16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21053db796e4440bb8901bca240edacd)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:46.186813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74133ae1-5e91-46e0-8220-083250c1db16 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:46.333635 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-74133ae1-5e91-46e0-8220-083250c1db16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21053db796e4440bb8901bca240edacd', 'request_id': 'req-74133ae1-5e91-46e0-8220-083250c1db16'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:46.461228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-74133ae1-5e91-46e0-8220-083250c1db16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21053db796e4440bb8901bca240edacd', 'request_id': 'req-74133ae1-5e91-46e0-8220-083250c1db16'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:46.475220 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-74133ae1-5e91-46e0-8220-083250c1db16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21053db796e4440bb8901bca240edacd', 'request_id': 'req-74133ae1-5e91-46e0-8220-083250c1db16'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:46.497320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-74133ae1-5e91-46e0-8220-083250c1db16 admin admin] The token cache is being invalidate because project 21053db796e4440bb8901bca240edacd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:46.497631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-74133ae1-5e91-46e0-8220-083250c1db16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:46.498918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-74133ae1-5e91-46e0-8220-083250c1db16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:46.500811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 981/1968] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:46 2022] DELETE /identity/v3/projects/21053db796e4440bb8901bca240edacd => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:46.507520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29213f-76d6-465b-92a7-c53c5461a68c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:46.508287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29213f-76d6-465b-92a7-c53c5461a68c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:46.508475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29213f-76d6-465b-92a7-c53c5461a68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:46.543616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ce29213f-76d6-465b-92a7-c53c5461a68c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:46.545820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ce29213f-76d6-465b-92a7-c53c5461a68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:46.644083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 988/1969] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:46.652978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efc8ce-0ea5-49e6-9e3e-997d4e8591d4 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:46.653278 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efc8ce-0ea5-49e6-9e3e-997d4e8591d4 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:46.653767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efc8ce-0ea5-49e6-9e3e-997d4e8591d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:46.723296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-29efc8ce-0ea5-49e6-9e3e-997d4e8591d4 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:46.724843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-29efc8ce-0ea5-49e6-9e3e-997d4e8591d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:46.737787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 982/1970] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:46.745517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29213f-76d6-465b-92a7-c53c5461a68c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:46.761408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29213f-76d6-465b-92a7-c53c5461a68c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:46.777053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398eca6-8e2a-475e-96fc-c6d6da588a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:46.777811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398eca6-8e2a-475e-96fc-c6d6da588a38 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:46.778108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398eca6-8e2a-475e-96fc-c6d6da588a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:46.778430 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398eca6-8e2a-475e-96fc-c6d6da588a38 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:46.779163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398eca6-8e2a-475e-96fc-c6d6da588a38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:05:46.779542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398eca6-8e2a-475e-96fc-c6d6da588a38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:46.781476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398eca6-8e2a-475e-96fc-c6d6da588a38 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:46.800452 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 989/1971] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:05:46.806909 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efc8ce-0ea5-49e6-9e3e-997d4e8591d4 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:46.820840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efc8ce-0ea5-49e6-9e3e-997d4e8591d4 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:46.826836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513667f-ddcb-4556-9394-914600eab37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:46.828973 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513667f-ddcb-4556-9394-914600eab37a admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:46.829465 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513667f-ddcb-4556-9394-914600eab37a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:46.830174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513667f-ddcb-4556-9394-914600eab37a admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:46.831550 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513667f-ddcb-4556-9394-914600eab37a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:46.835369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513667f-ddcb-4556-9394-914600eab37a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:47.051446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398eca6-8e2a-475e-96fc-c6d6da588a38 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:47.053994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-0398eca6-8e2a-475e-96fc-c6d6da588a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:47.068270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 983/1972] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:46 2022] POST /identity/v3/projects => generated 377 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:05:47.072878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398eca6-8e2a-475e-96fc-c6d6da588a38 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:47.074936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513667f-ddcb-4556-9394-914600eab37a admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:47.131806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:47.160386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:47.191283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a8327-24b8-436e-b1e8-fc9ce7db7c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:47.193962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a8327-24b8-436e-b1e8-fc9ce7db7c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:47.194216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a8327-24b8-436e-b1e8-fc9ce7db7c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:47.194915 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a8327-24b8-436e-b1e8-fc9ce7db7c35 admin admin] PATH_INFO: `/v3/users/6ca991be6007459eaf7a4f172ead0f85` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:47.197281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a8327-24b8-436e-b1e8-fc9ce7db7c35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ca991be6007459eaf7a4f172ead0f85)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:47.201560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a8327-24b8-436e-b1e8-fc9ce7db7c35 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:47.210821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513667f-ddcb-4556-9394-914600eab37a admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:47.231032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83777394-3c75-4457-bff0-78ca25698a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:47.231830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777394-3c75-4457-bff0-78ca25698a48 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:47.232292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777394-3c75-4457-bff0-78ca25698a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:47.233524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777394-3c75-4457-bff0-78ca25698a48 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:47.234564 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83777394-3c75-4457-bff0-78ca25698a48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:47.246452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83777394-3c75-4457-bff0-78ca25698a48 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:47.366155 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 984/1973] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:47 2022] POST /identity/v3/users => generated 289 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:47.370943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83777394-3c75-4457-bff0-78ca25698a48 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:47.386897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83777394-3c75-4457-bff0-78ca25698a48 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:47.396441 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bcdd5-1959-4949-ac42-0747b56bfd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:47.397731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bcdd5-1959-4949-ac42-0747b56bfd1f admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:47.398278 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bcdd5-1959-4949-ac42-0747b56bfd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:47.398844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bcdd5-1959-4949-ac42-0747b56bfd1f admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:47.400286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bcdd5-1959-4949-ac42-0747b56bfd1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:47.403733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bcdd5-1959-4949-ac42-0747b56bfd1f admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:47.422552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 985/1974] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:47.427605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bcdd5-1959-4949-ac42-0747b56bfd1f admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:47.440136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bcdd5-1959-4949-ac42-0747b56bfd1f admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:47.445532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d4cd0-d961-4f06-a2a6-7b92eefabbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:47.446252 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d4cd0-d961-4f06-a2a6-7b92eefabbf8 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:47.446580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d4cd0-d961-4f06-a2a6-7b92eefabbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:47.446863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d4cd0-d961-4f06-a2a6-7b92eefabbf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:47.447611 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d4cd0-d961-4f06-a2a6-7b92eefabbf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:47.449411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d4cd0-d961-4f06-a2a6-7b92eefabbf8 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:47.463604 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 986/1975] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:47.469333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d4cd0-d961-4f06-a2a6-7b92eefabbf8 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:47.479483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2c4a8327-24b8-436e-b1e8-fc9ce7db7c35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ca991be6007459eaf7a4f172ead0f85'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:47.480193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2c4a8327-24b8-436e-b1e8-fc9ce7db7c35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ca991be6007459eaf7a4f172ead0f85'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:47.482255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d4cd0-d961-4f06-a2a6-7b92eefabbf8 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:47.490564 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da0e2e-a989-4d8f-925b-ec87dc15f75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:47.491612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da0e2e-a989-4d8f-925b-ec87dc15f75e admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:47.491909 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da0e2e-a989-4d8f-925b-ec87dc15f75e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:47.492256 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da0e2e-a989-4d8f-925b-ec87dc15f75e admin admin] PATH_INFO: `/v3/projects/e0551a17b94a420186fe8f6d100b37f6/users/d9f829be64744329ad991576e69512f5/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:47.501074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2c4a8327-24b8-436e-b1e8-fc9ce7db7c35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ca991be6007459eaf7a4f172ead0f85'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:47.515746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da0e2e-a989-4d8f-925b-ec87dc15f75e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0551a17b94a420186fe8f6d100b37f6, user_id=d9f829be64744329ad991576e69512f5, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:47.519179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da0e2e-a989-4d8f-925b-ec87dc15f75e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:47.520514 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2c4a8327-24b8-436e-b1e8-fc9ce7db7c35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ca991be6007459eaf7a4f172ead0f85'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:47.673422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2c4a8327-24b8-436e-b1e8-fc9ce7db7c35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ca991be6007459eaf7a4f172ead0f85'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:47.683526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 990/1976] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:47 2022] DELETE /identity/v3/users/6ca991be6007459eaf7a4f172ead0f85 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:47.693749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb537ed3-589a-4863-8152-e7d3b973c644 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:47.694260 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb537ed3-589a-4863-8152-e7d3b973c644 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:47.694762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb537ed3-589a-4863-8152-e7d3b973c644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:47.703972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 987/1977] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:47 2022] PUT /identity/v3/projects/e0551a17b94a420186fe8f6d100b37f6/users/d9f829be64744329ad991576e69512f5/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:47.731276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13a9ed-4a95-481d-a59d-381f50558e14 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:47.731629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13a9ed-4a95-481d-a59d-381f50558e14 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:47.731984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-fb537ed3-589a-4863-8152-e7d3b973c644 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:47.732176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13a9ed-4a95-481d-a59d-381f50558e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:47.733592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fb537ed3-589a-4863-8152-e7d3b973c644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:47.819019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-da13a9ed-4a95-481d-a59d-381f50558e14 None None] MFA Rules not processed for user `d9f829be64744329ad991576e69512f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:47.822396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-da13a9ed-4a95-481d-a59d-381f50558e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:47.867606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 991/1978] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:47.882026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb537ed3-589a-4863-8152-e7d3b973c644 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:47.888034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fb537ed3-589a-4863-8152-e7d3b973c644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:47.986180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 988/1979] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:48.016690 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13a9ed-4a95-481d-a59d-381f50558e14 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:48.018403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-da13a9ed-4a95-481d-a59d-381f50558e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:48.083966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb537ed3-589a-4863-8152-e7d3b973c644 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:48.106505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bdb76-90d4-4bfb-bad4-7ec1256d9fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:48.106505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bdb76-90d4-4bfb-bad4-7ec1256d9fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:48.106505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bdb76-90d4-4bfb-bad4-7ec1256d9fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:48.106505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bdb76-90d4-4bfb-bad4-7ec1256d9fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:48.132966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bdb76-90d4-4bfb-bad4-7ec1256d9fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:48.132966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bdb76-90d4-4bfb-bad4-7ec1256d9fe9 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:48.168932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:48.183115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13a9ed-4a95-481d-a59d-381f50558e14 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:48.190881 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 992/1980] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:48.194308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b94c7-3baa-48c1-8b55-efb13fa0e92b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:48.195145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b94c7-3baa-48c1-8b55-efb13fa0e92b service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:48.195437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b94c7-3baa-48c1-8b55-efb13fa0e92b service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:48.195793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b94c7-3baa-48c1-8b55-efb13fa0e92b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:48.227762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b94c7-3baa-48c1-8b55-efb13fa0e92b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:48.229948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b94c7-3baa-48c1-8b55-efb13fa0e92b service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:48.257288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 989/1981] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:49.620245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bdb76-90d4-4bfb-bad4-7ec1256d9fe9 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:49.621929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-087bdb76-90d4-4bfb-bad4-7ec1256d9fe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:49.643280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bdb76-90d4-4bfb-bad4-7ec1256d9fe9 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:49.649640 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:49.650747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:49.651043 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:49.651351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] PATH_INFO: `/v3/projects/1aaea95be92540759ce95f8af5c4f4b9` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:49.653845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aaea95be92540759ce95f8af5c4f4b9)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:49.656883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:49.862685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aaea95be92540759ce95f8af5c4f4b9', 'request_id': 'req-f2eb8d06-e8f4-494c-a920-ad9e68a54282'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:49.875873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aaea95be92540759ce95f8af5c4f4b9', 'request_id': 'req-f2eb8d06-e8f4-494c-a920-ad9e68a54282'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:49.886630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aaea95be92540759ce95f8af5c4f4b9', 'request_id': 'req-f2eb8d06-e8f4-494c-a920-ad9e68a54282'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:50.029275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] The token cache is being invalidate because project 1aaea95be92540759ce95f8af5c4f4b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:50.030902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:50.032467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:50.036586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 993/1982] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:49 2022] DELETE /identity/v3/projects/1aaea95be92540759ce95f8af5c4f4b9 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:50.041289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b94c7-3baa-48c1-8b55-efb13fa0e92b service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:50.056189 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b94c7-3baa-48c1-8b55-efb13fa0e92b service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:50.074245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:50.142941 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:50.143998 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:50.144184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:50.144474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] PATH_INFO: `/v3/users/5ad8619858734853825fafa1ba6cd021` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:50.145946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ad8619858734853825fafa1ba6cd021)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:50.148142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:50.436503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ad8619858734853825fafa1ba6cd021'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:50.438156 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ad8619858734853825fafa1ba6cd021'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:50.439278 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ad8619858734853825fafa1ba6cd021'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:50.547479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ad8619858734853825fafa1ba6cd021'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:50.567150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ad8619858734853825fafa1ba6cd021'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:50.587463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 990/1983] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:50 2022] DELETE /identity/v3/users/5ad8619858734853825fafa1ba6cd021 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:51.543039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:51.543039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:51.562657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb8d06-e8f4-494c-a920-ad9e68a54282 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:51.695873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdfeda-8657-4678-a600-5b43c85074ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:51.695873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdfeda-8657-4678-a600-5b43c85074ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:51.695873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdfeda-8657-4678-a600-5b43c85074ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:51.695873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdfeda-8657-4678-a600-5b43c85074ea admin admin] PATH_INFO: `/v3/projects/21ca773a37074b7987cb0d679e6ad13f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:51.696566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdfeda-8657-4678-a600-5b43c85074ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=21ca773a37074b7987cb0d679e6ad13f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:51.696566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdfeda-8657-4678-a600-5b43c85074ea admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:51.935574 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:51.937734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:51.953980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a5bdfeda-8657-4678-a600-5b43c85074ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21ca773a37074b7987cb0d679e6ad13f', 'request_id': 'req-a5bdfeda-8657-4678-a600-5b43c85074ea'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:51.967469 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f09656-3a6c-4b67-b64f-f1457fbb2032 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:51.979347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a5bdfeda-8657-4678-a600-5b43c85074ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21ca773a37074b7987cb0d679e6ad13f', 'request_id': 'req-a5bdfeda-8657-4678-a600-5b43c85074ea'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:51.989066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a5bdfeda-8657-4678-a600-5b43c85074ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21ca773a37074b7987cb0d679e6ad13f', 'request_id': 'req-a5bdfeda-8657-4678-a600-5b43c85074ea'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:52.149201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ae2b8-2823-4e5a-b01c-358549c8118f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:52.150878 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ae2b8-2823-4e5a-b01c-358549c8118f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:52.151540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ae2b8-2823-4e5a-b01c-358549c8118f admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:52.152144 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ae2b8-2823-4e5a-b01c-358549c8118f admin admin] PATH_INFO: `/v3/users/a878b057245340e6990c39ccddfd4bf4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:52.165076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ae2b8-2823-4e5a-b01c-358549c8118f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a878b057245340e6990c39ccddfd4bf4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:52.169482 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ae2b8-2823-4e5a-b01c-358549c8118f admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:52.210674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a5bdfeda-8657-4678-a600-5b43c85074ea admin admin] The token cache is being invalidate because project 21ca773a37074b7987cb0d679e6ad13f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:52.210674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a5bdfeda-8657-4678-a600-5b43c85074ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:52.210674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a5bdfeda-8657-4678-a600-5b43c85074ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:52.210674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 994/1984] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:51 2022] DELETE /identity/v3/projects/21ca773a37074b7987cb0d679e6ad13f => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:52.215138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceff54d-dbd5-4aab-911a-ec4dd6822a01 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:52.215643 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceff54d-dbd5-4aab-911a-ec4dd6822a01 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:52.216035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceff54d-dbd5-4aab-911a-ec4dd6822a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:52.257965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-cceff54d-dbd5-4aab-911a-ec4dd6822a01 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:52.260364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-cceff54d-dbd5-4aab-911a-ec4dd6822a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:52.273231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 995/1985] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:05:52.285131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdef77-b8f8-49e0-a86d-26eb3cf2a077 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:52.285131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdef77-b8f8-49e0-a86d-26eb3cf2a077 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:52.285131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdef77-b8f8-49e0-a86d-26eb3cf2a077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:52.302458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:52.328060 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-13bdef77-b8f8-49e0-a86d-26eb3cf2a077 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:52.329377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-13bdef77-b8f8-49e0-a86d-26eb3cf2a077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:52.364587 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 996/1986] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:52.370559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdef77-b8f8-49e0-a86d-26eb3cf2a077 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:52.398545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdef77-b8f8-49e0-a86d-26eb3cf2a077 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:52.412216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56391aa3-25a2-4444-abf5-60668ff937e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:52.413459 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56391aa3-25a2-4444-abf5-60668ff937e5 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:52.414277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56391aa3-25a2-4444-abf5-60668ff937e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:52.414979 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56391aa3-25a2-4444-abf5-60668ff937e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:52.415770 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56391aa3-25a2-4444-abf5-60668ff937e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:05:52.416593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56391aa3-25a2-4444-abf5-60668ff937e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:52.419509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56391aa3-25a2-4444-abf5-60668ff937e5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:52.439323 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 997/1987] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:05:52.445748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:52.453042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56391aa3-25a2-4444-abf5-60668ff937e5 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:52.479912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56391aa3-25a2-4444-abf5-60668ff937e5 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:52.480179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-810ae2b8-2823-4e5a-b01c-358549c8118f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a878b057245340e6990c39ccddfd4bf4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:52.482228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-810ae2b8-2823-4e5a-b01c-358549c8118f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a878b057245340e6990c39ccddfd4bf4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:52.487894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-810ae2b8-2823-4e5a-b01c-358549c8118f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a878b057245340e6990c39ccddfd4bf4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:52.497586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa02a7e-575e-469e-beab-467fd1530243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:52.499153 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa02a7e-575e-469e-beab-467fd1530243 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:52.499592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa02a7e-575e-469e-beab-467fd1530243 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:52.500541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa02a7e-575e-469e-beab-467fd1530243 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:52.516343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa02a7e-575e-469e-beab-467fd1530243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:52.520531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa02a7e-575e-469e-beab-467fd1530243 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:52.644209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-810ae2b8-2823-4e5a-b01c-358549c8118f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a878b057245340e6990c39ccddfd4bf4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:52.657694 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 998/1988] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:52 2022] POST /identity/v3/projects => generated 363 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:52.659557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa02a7e-575e-469e-beab-467fd1530243 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:52.661801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6aa02a7e-575e-469e-beab-467fd1530243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:52.668437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-810ae2b8-2823-4e5a-b01c-358549c8118f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a878b057245340e6990c39ccddfd4bf4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:52.704800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 991/1989] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:51 2022] DELETE /identity/v3/users/a878b057245340e6990c39ccddfd4bf4 => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:52.710805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7993c45-e5bf-4912-80a9-d79373d7b452 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:52.711162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7993c45-e5bf-4912-80a9-d79373d7b452 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:52.711711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7993c45-e5bf-4912-80a9-d79373d7b452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:52.750518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c7993c45-e5bf-4912-80a9-d79373d7b452 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:52.750518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c7993c45-e5bf-4912-80a9-d79373d7b452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:52.796956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa02a7e-575e-469e-beab-467fd1530243 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:52.884431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 992/1990] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:52.898696 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7993c45-e5bf-4912-80a9-d79373d7b452 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:52.902228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c7993c45-e5bf-4912-80a9-d79373d7b452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:52.908751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e227e91-d6bb-42eb-81cc-65be9fedf6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:52.909927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e227e91-d6bb-42eb-81cc-65be9fedf6ea admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:52.911467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e227e91-d6bb-42eb-81cc-65be9fedf6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:52.912166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e227e91-d6bb-42eb-81cc-65be9fedf6ea admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:52.913608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e227e91-d6bb-42eb-81cc-65be9fedf6ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:52.918232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e227e91-d6bb-42eb-81cc-65be9fedf6ea admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:52.988765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7993c45-e5bf-4912-80a9-d79373d7b452 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:52.994347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c168cb0-d366-48a2-8c11-b0f5d7e1a082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:52.995106 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c168cb0-d366-48a2-8c11-b0f5d7e1a082 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:52.995399 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c168cb0-d366-48a2-8c11-b0f5d7e1a082 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:52.995685 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c168cb0-d366-48a2-8c11-b0f5d7e1a082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:53.013337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c168cb0-d366-48a2-8c11-b0f5d7e1a082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:53.015929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c168cb0-d366-48a2-8c11-b0f5d7e1a082 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:53.032763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 993/1991] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:53.040728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 999/1992] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:52 2022] POST /identity/v3/users => generated 282 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:53.046857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e227e91-d6bb-42eb-81cc-65be9fedf6ea admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:53.106671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e227e91-d6bb-42eb-81cc-65be9fedf6ea admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:53.127111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21743b02-e8e8-4f19-87ec-eeace3023567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:53.130301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21743b02-e8e8-4f19-87ec-eeace3023567 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:53.142996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21743b02-e8e8-4f19-87ec-eeace3023567 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:53.144826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21743b02-e8e8-4f19-87ec-eeace3023567 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:53.147261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21743b02-e8e8-4f19-87ec-eeace3023567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:53.164222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21743b02-e8e8-4f19-87ec-eeace3023567 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:53.184395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1000/1993] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:53.189357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c168cb0-d366-48a2-8c11-b0f5d7e1a082 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:53.204995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c168cb0-d366-48a2-8c11-b0f5d7e1a082 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:53.212758 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bbdeb-dbe3-48a2-9dde-242d2d7ad568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:53.213543 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bbdeb-dbe3-48a2-9dde-242d2d7ad568 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:53.213942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bbdeb-dbe3-48a2-9dde-242d2d7ad568 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:53.215092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bbdeb-dbe3-48a2-9dde-242d2d7ad568 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:53.219077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bbdeb-dbe3-48a2-9dde-242d2d7ad568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:53.219077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bbdeb-dbe3-48a2-9dde-242d2d7ad568 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:53.234481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 994/1994] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:53.250183 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21743b02-e8e8-4f19-87ec-eeace3023567 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:53.288576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21743b02-e8e8-4f19-87ec-eeace3023567 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:53.304079 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eef2f2-706f-4d42-96e1-2baff6b370e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:53.305124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eef2f2-706f-4d42-96e1-2baff6b370e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:53.305425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eef2f2-706f-4d42-96e1-2baff6b370e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:53.305708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eef2f2-706f-4d42-96e1-2baff6b370e2 admin admin] PATH_INFO: `/v3/projects/cbde31b4a73b4dd59091f34c6d46dcb8/users/8e16960c854543a8bdf141e278fac0c2/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:53.373173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eef2f2-706f-4d42-96e1-2baff6b370e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbde31b4a73b4dd59091f34c6d46dcb8, user_id=8e16960c854543a8bdf141e278fac0c2, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:53.376385 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eef2f2-706f-4d42-96e1-2baff6b370e2 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:53.494383 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1001/1995] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:53 2022] PUT /identity/v3/projects/cbde31b4a73b4dd59091f34c6d46dcb8/users/8e16960c854543a8bdf141e278fac0c2/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:53.516390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d25f6a-1281-4995-b63f-a872240b1d2d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:53.516390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d25f6a-1281-4995-b63f-a872240b1d2d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:53.516752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d25f6a-1281-4995-b63f-a872240b1d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:53.581802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-37d25f6a-1281-4995-b63f-a872240b1d2d None None] MFA Rules not processed for user `8e16960c854543a8bdf141e278fac0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:53.583819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-37d25f6a-1281-4995-b63f-a872240b1d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:53.634005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:53.696820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 995/1996] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:53 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:53.702895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eef2f2-706f-4d42-96e1-2baff6b370e2 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:53.733497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d25f6a-1281-4995-b63f-a872240b1d2d None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:53.734617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-37d25f6a-1281-4995-b63f-a872240b1d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:53.801532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:53.888170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eef2f2-706f-4d42-96e1-2baff6b370e2 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:53.895528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230cf6db-a927-4bc5-83b4-a44a911583a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:53.896508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cf6db-a927-4bc5-83b4-a44a911583a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:53.897052 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cf6db-a927-4bc5-83b4-a44a911583a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:53.897482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cf6db-a927-4bc5-83b4-a44a911583a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:53.898435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230cf6db-a927-4bc5-83b4-a44a911583a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:53.910270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d25f6a-1281-4995-b63f-a872240b1d2d None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:53.911197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230cf6db-a927-4bc5-83b4-a44a911583a4 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:53.919055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:53.919435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:53.919578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:53.919917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] PATH_INFO: `/v3/projects/2f068693208345f88cf658bc625fb4e1` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:53.921633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f068693208345f88cf658bc625fb4e1)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:53.924082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:54.061057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1002/1997] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:53 2022] POST /identity/v3/projects => generated 365 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:54.063610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230cf6db-a927-4bc5-83b4-a44a911583a4 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:54.137577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f068693208345f88cf658bc625fb4e1', 'request_id': 'req-a65f1501-225d-4dcd-916b-806367652167'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:54.178567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230cf6db-a927-4bc5-83b4-a44a911583a4 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:54.229645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f068693208345f88cf658bc625fb4e1', 'request_id': 'req-a65f1501-225d-4dcd-916b-806367652167'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:54.251778 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f068693208345f88cf658bc625fb4e1', 'request_id': 'req-a65f1501-225d-4dcd-916b-806367652167'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:54.282235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1fb5b-6795-4921-8b0b-066f7e7a444b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:54.285735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1fb5b-6795-4921-8b0b-066f7e7a444b admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:54.287632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1fb5b-6795-4921-8b0b-066f7e7a444b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:54.287940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1fb5b-6795-4921-8b0b-066f7e7a444b admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:54.288696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1fb5b-6795-4921-8b0b-066f7e7a444b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:54.297124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1fb5b-6795-4921-8b0b-066f7e7a444b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:54.297982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] The token cache is being invalidate because project 2f068693208345f88cf658bc625fb4e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:54.298286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:54.300687 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:54.303892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 996/1998] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:53 2022] DELETE /identity/v3/projects/2f068693208345f88cf658bc625fb4e1 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:54.309184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:54.315030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:54.334584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f1501-225d-4dcd-916b-806367652167 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:54.344087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306137b-39f0-4562-bb41-b7196d6c6997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:54.345639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306137b-39f0-4562-bb41-b7196d6c6997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:54.346262 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306137b-39f0-4562-bb41-b7196d6c6997 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:54.346941 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306137b-39f0-4562-bb41-b7196d6c6997 admin admin] PATH_INFO: `/v3/users/2803acbf41174a5ab942538477236ae8` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:54.349314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306137b-39f0-4562-bb41-b7196d6c6997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2803acbf41174a5ab942538477236ae8)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:54.353135 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306137b-39f0-4562-bb41-b7196d6c6997 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:54.399502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1003/1999] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:54 2022] POST /identity/v3/users => generated 289 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:05:54.405366 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1fb5b-6795-4921-8b0b-066f7e7a444b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:54.408596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-83e1fb5b-6795-4921-8b0b-066f7e7a444b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:54.428350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1fb5b-6795-4921-8b0b-066f7e7a444b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:54.434885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e5cd8-c229-446a-bb56-781362401601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:54.437656 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e5cd8-c229-446a-bb56-781362401601 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:54.438172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e5cd8-c229-446a-bb56-781362401601 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:54.438580 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e5cd8-c229-446a-bb56-781362401601 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:54.439708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e5cd8-c229-446a-bb56-781362401601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:54.442456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e5cd8-c229-446a-bb56-781362401601 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:54.454920 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1004/2000] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:05:54.459980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e5cd8-c229-446a-bb56-781362401601 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:54.471416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e5cd8-c229-446a-bb56-781362401601 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:54.478045 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11d038-8510-4558-aa2a-e5eaa87bc086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:54.479606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11d038-8510-4558-aa2a-e5eaa87bc086 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:54.480816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11d038-8510-4558-aa2a-e5eaa87bc086 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:54.480816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11d038-8510-4558-aa2a-e5eaa87bc086 admin admin] PATH_INFO: `/v3/projects/affa82ed045f4463b5f29a9dadfbbf06/users/a2b2ff88a0db4d84a118f1f0c51868bf/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:54.501260 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11d038-8510-4558-aa2a-e5eaa87bc086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=affa82ed045f4463b5f29a9dadfbbf06, user_id=a2b2ff88a0db4d84a118f1f0c51868bf, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:54.506387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11d038-8510-4558-aa2a-e5eaa87bc086 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:54.580634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7306137b-39f0-4562-bb41-b7196d6c6997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2803acbf41174a5ab942538477236ae8'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:54.581645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7306137b-39f0-4562-bb41-b7196d6c6997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2803acbf41174a5ab942538477236ae8'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:54.582786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7306137b-39f0-4562-bb41-b7196d6c6997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2803acbf41174a5ab942538477236ae8'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:54.608767 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1005/2001] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:54 2022] PUT /identity/v3/projects/affa82ed045f4463b5f29a9dadfbbf06/users/a2b2ff88a0db4d84a118f1f0c51868bf/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:54.615425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11d038-8510-4558-aa2a-e5eaa87bc086 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:54.617843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fa11d038-8510-4558-aa2a-e5eaa87bc086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:54.637463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:54.717669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11d038-8510-4558-aa2a-e5eaa87bc086 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:54.726958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1baed-44e4-4623-8105-75be33b449fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:54.728081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1baed-44e4-4623-8105-75be33b449fd admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:54.729193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1baed-44e4-4623-8105-75be33b449fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:54.729652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1baed-44e4-4623-8105-75be33b449fd admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:54.731235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1baed-44e4-4623-8105-75be33b449fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:54.733883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1baed-44e4-4623-8105-75be33b449fd admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:54.739356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7306137b-39f0-4562-bb41-b7196d6c6997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2803acbf41174a5ab942538477236ae8'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:54.741626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:54.750824 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1006/2002] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:54.755883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1baed-44e4-4623-8105-75be33b449fd admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:54.766846 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7306137b-39f0-4562-bb41-b7196d6c6997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2803acbf41174a5ab942538477236ae8'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:54.771157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1baed-44e4-4623-8105-75be33b449fd admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:54.778116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc1cab-d5a5-4864-9a15-28f7cec481f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:54.778746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc1cab-d5a5-4864-9a15-28f7cec481f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:54.779105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc1cab-d5a5-4864-9a15-28f7cec481f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:54.782511 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc1cab-d5a5-4864-9a15-28f7cec481f5 admin admin] PATH_INFO: `/v3/projects/affa82ed045f4463b5f29a9dadfbbf06/users/a2b2ff88a0db4d84a118f1f0c51868bf/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:54.783347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc1cab-d5a5-4864-9a15-28f7cec481f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=affa82ed045f4463b5f29a9dadfbbf06, user_id=a2b2ff88a0db4d84a118f1f0c51868bf, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:54.787471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 997/2003] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:54 2022] DELETE /identity/v3/users/2803acbf41174a5ab942538477236ae8 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:54.787932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc1cab-d5a5-4864-9a15-28f7cec481f5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:54.825680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1007/2004] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:54 2022] PUT /identity/v3/projects/affa82ed045f4463b5f29a9dadfbbf06/users/a2b2ff88a0db4d84a118f1f0c51868bf/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:54.832240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2f5d6-4ee8-4bca-bbe6-5a606ed5855b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:54.832922 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2f5d6-4ee8-4bca-bbe6-5a606ed5855b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:54.834475 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2f5d6-4ee8-4bca-bbe6-5a606ed5855b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:54.898682 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-d0c2f5d6-4ee8-4bca-bbe6-5a606ed5855b None None] MFA Rules not processed for user `a2b2ff88a0db4d84a118f1f0c51868bf`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:54.901311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d0c2f5d6-4ee8-4bca-bbe6-5a606ed5855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:55.039046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 998/2005] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:54 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:55.056625 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc1cab-d5a5-4864-9a15-28f7cec481f5 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:55.059562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6bdc1cab-d5a5-4864-9a15-28f7cec481f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:55.175534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc1cab-d5a5-4864-9a15-28f7cec481f5 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:55.182664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3980a-6bbb-4f9f-a011-b0128a776450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:55.183833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3980a-6bbb-4f9f-a011-b0128a776450 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:55.184147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3980a-6bbb-4f9f-a011-b0128a776450 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:55.184417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3980a-6bbb-4f9f-a011-b0128a776450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:55.197678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3980a-6bbb-4f9f-a011-b0128a776450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:55.200873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3980a-6bbb-4f9f-a011-b0128a776450 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:55.227131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1008/2006] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:55 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:55.487456 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2f5d6-4ee8-4bca-bbe6-5a606ed5855b None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:55.506357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2f5d6-4ee8-4bca-bbe6-5a606ed5855b None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:55.516145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c517e-bace-495d-8760-7391361be1ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:55.516748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c517e-bace-495d-8760-7391361be1ff service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:55.517216 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c517e-bace-495d-8760-7391361be1ff service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:55.517588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c517e-bace-495d-8760-7391361be1ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:55.520032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-be2c517e-bace-495d-8760-7391361be1ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:55.626701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c517e-bace-495d-8760-7391361be1ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:55.628120 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c517e-bace-495d-8760-7391361be1ff service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:55.642951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 999/2007] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:55 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:05:55.666673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3980a-6bbb-4f9f-a011-b0128a776450 service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:55.666673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-33e3980a-6bbb-4f9f-a011-b0128a776450 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:55.829404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3980a-6bbb-4f9f-a011-b0128a776450 service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:55.836100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:55.837132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:55.837437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:55.837690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] PATH_INFO: `/v3/projects/97c244082b9d4c3dbbc12b16abb6c971` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:55.839339 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] RBAC: Authorizing `identity:delete_project(project_id=97c244082b9d4c3dbbc12b16abb6c971)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:55.841324 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:56.027731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:56.041112 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97c244082b9d4c3dbbc12b16abb6c971', 'request_id': 'req-9f7ba035-9c86-493d-8b24-17327ed0e55a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:56.051332 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97c244082b9d4c3dbbc12b16abb6c971', 'request_id': 'req-9f7ba035-9c86-493d-8b24-17327ed0e55a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:56.059389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97c244082b9d4c3dbbc12b16abb6c971', 'request_id': 'req-9f7ba035-9c86-493d-8b24-17327ed0e55a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:56.272448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] The token cache is being invalidate because project 97c244082b9d4c3dbbc12b16abb6c971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:56.272448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:56.272448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:56.272448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1009/2008] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:55 2022] DELETE /identity/v3/projects/97c244082b9d4c3dbbc12b16abb6c971 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:56.274798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c517e-bace-495d-8760-7391361be1ff service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:56.274798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c517e-bace-495d-8760-7391361be1ff service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:56.397839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f812660-20f3-4f51-ab8f-c4dbecaa6bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:56.399667 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f812660-20f3-4f51-ab8f-c4dbecaa6bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:56.400062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f812660-20f3-4f51-ab8f-c4dbecaa6bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:56.400355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f812660-20f3-4f51-ab8f-c4dbecaa6bd2 admin admin] PATH_INFO: `/v3/users/a0f35828d260447593c7d3602a1c2cf7` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:56.401885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f812660-20f3-4f51-ab8f-c4dbecaa6bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f35828d260447593c7d3602a1c2cf7)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:56.404125 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f812660-20f3-4f51-ab8f-c4dbecaa6bd2 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:56.628858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-9f812660-20f3-4f51-ab8f-c4dbecaa6bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f35828d260447593c7d3602a1c2cf7'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:56.629945 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-9f812660-20f3-4f51-ab8f-c4dbecaa6bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f35828d260447593c7d3602a1c2cf7'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:56.630789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-9f812660-20f3-4f51-ab8f-c4dbecaa6bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f35828d260447593c7d3602a1c2cf7'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:56.711461 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-9f812660-20f3-4f51-ab8f-c4dbecaa6bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f35828d260447593c7d3602a1c2cf7'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:56.730876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-9f812660-20f3-4f51-ab8f-c4dbecaa6bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f35828d260447593c7d3602a1c2cf7'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:56.774950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1000/2009] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:05:56 2022] DELETE /identity/v3/users/a0f35828d260447593c7d3602a1c2cf7 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:56.777237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:57.351400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:57.352389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:57.382445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ba035-9c86-493d-8b24-17327ed0e55a admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:57.476912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed21b9-5f99-4956-b3dc-5043b198627a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:57.478403 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed21b9-5f99-4956-b3dc-5043b198627a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:57.478403 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed21b9-5f99-4956-b3dc-5043b198627a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:57.479002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed21b9-5f99-4956-b3dc-5043b198627a admin admin] PATH_INFO: `/v3/projects/3f8287dbf5554b428dd8fb8a766601aa` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:57.482791 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed21b9-5f99-4956-b3dc-5043b198627a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f8287dbf5554b428dd8fb8a766601aa)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:57.483360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed21b9-5f99-4956-b3dc-5043b198627a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:57.719894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-05ed21b9-5f99-4956-b3dc-5043b198627a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f8287dbf5554b428dd8fb8a766601aa', 'request_id': 'req-05ed21b9-5f99-4956-b3dc-5043b198627a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:57.750660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-05ed21b9-5f99-4956-b3dc-5043b198627a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f8287dbf5554b428dd8fb8a766601aa', 'request_id': 'req-05ed21b9-5f99-4956-b3dc-5043b198627a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:57.767345 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-05ed21b9-5f99-4956-b3dc-5043b198627a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f8287dbf5554b428dd8fb8a766601aa', 'request_id': 'req-05ed21b9-5f99-4956-b3dc-5043b198627a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:57.865828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-05ed21b9-5f99-4956-b3dc-5043b198627a admin admin] The token cache is being invalidate because project 3f8287dbf5554b428dd8fb8a766601aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:05:57.866130 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-05ed21b9-5f99-4956-b3dc-5043b198627a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:57.866833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-05ed21b9-5f99-4956-b3dc-5043b198627a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:05:57.868409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1010/2010] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:05:57 2022] DELETE /identity/v3/projects/3f8287dbf5554b428dd8fb8a766601aa => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:05:57.874450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02186483-f0d2-4012-97bd-0293e8a97613 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:57.875088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02186483-f0d2-4012-97bd-0293e8a97613 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:57.875088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02186483-f0d2-4012-97bd-0293e8a97613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:57.905337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-02186483-f0d2-4012-97bd-0293e8a97613 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:57.906904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-02186483-f0d2-4012-97bd-0293e8a97613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:57.976681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1001/2011] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:57.987713 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aceb951-39d3-4e29-916e-b0d63e7cf87b None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:57.988102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aceb951-39d3-4e29-916e-b0d63e7cf87b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:57.988459 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aceb951-39d3-4e29-916e-b0d63e7cf87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:58.014532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-7aceb951-39d3-4e29-916e-b0d63e7cf87b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:58.016500 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7aceb951-39d3-4e29-916e-b0d63e7cf87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:58.026577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1011/2012] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:58.032627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02186483-f0d2-4012-97bd-0293e8a97613 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:58.043826 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02186483-f0d2-4012-97bd-0293e8a97613 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:58.050087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a4031-a8d0-41ad-90e7-3fda34e26ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:58.050985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a4031-a8d0-41ad-90e7-3fda34e26ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:58.051276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a4031-a8d0-41ad-90e7-3fda34e26ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:58.051673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a4031-a8d0-41ad-90e7-3fda34e26ee9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:58.052467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a4031-a8d0-41ad-90e7-3fda34e26ee9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:05:58.052779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a4031-a8d0-41ad-90e7-3fda34e26ee9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:58.054492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a4031-a8d0-41ad-90e7-3fda34e26ee9 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:58.074350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1002/2013] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:05:58.080795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aceb951-39d3-4e29-916e-b0d63e7cf87b None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:58.096784 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aceb951-39d3-4e29-916e-b0d63e7cf87b None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:58.103554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8375b-56ef-4b24-9dc1-57634f5aa008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:58.105231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8375b-56ef-4b24-9dc1-57634f5aa008 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:58.105231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8375b-56ef-4b24-9dc1-57634f5aa008 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:58.105231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8375b-56ef-4b24-9dc1-57634f5aa008 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:58.105844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8375b-56ef-4b24-9dc1-57634f5aa008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:58.108053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8375b-56ef-4b24-9dc1-57634f5aa008 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:58.269495 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1012/2014] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:05:58 2022] POST /identity/v3/projects => generated 371 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:05:58.270400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a4031-a8d0-41ad-90e7-3fda34e26ee9 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:58.283829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a4031-a8d0-41ad-90e7-3fda34e26ee9 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:58.404843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5a2fa-29a4-4a70-bc71-01ac9ec49cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:58.406513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5a2fa-29a4-4a70-bc71-01ac9ec49cfe admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:58.407115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5a2fa-29a4-4a70-bc71-01ac9ec49cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:58.407730 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5a2fa-29a4-4a70-bc71-01ac9ec49cfe admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:58.409131 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5a2fa-29a4-4a70-bc71-01ac9ec49cfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:58.412997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5a2fa-29a4-4a70-bc71-01ac9ec49cfe admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:58.428536 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:58.562789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1003/2015] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:05:58 2022] POST /identity/v3/users => generated 286 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:05:58.567046 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8375b-56ef-4b24-9dc1-57634f5aa008 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:58.586005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8375b-56ef-4b24-9dc1-57634f5aa008 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:58.592597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0410d31-3a20-4875-bd6c-83b332bd6019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:58.593851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0410d31-3a20-4875-bd6c-83b332bd6019 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:58.594355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0410d31-3a20-4875-bd6c-83b332bd6019 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:58.594922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0410d31-3a20-4875-bd6c-83b332bd6019 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:58.595961 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0410d31-3a20-4875-bd6c-83b332bd6019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:58.598078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0410d31-3a20-4875-bd6c-83b332bd6019 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:58.615553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1013/2016] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:58.616677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5a2fa-29a4-4a70-bc71-01ac9ec49cfe admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:58.648261 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5a2fa-29a4-4a70-bc71-01ac9ec49cfe admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:58.654851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29e6bb-4cfd-44d5-95c9-c560ba030d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:58.655981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29e6bb-4cfd-44d5-95c9-c560ba030d9b admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:58.656402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29e6bb-4cfd-44d5-95c9-c560ba030d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:58.656793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29e6bb-4cfd-44d5-95c9-c560ba030d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:58.658139 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29e6bb-4cfd-44d5-95c9-c560ba030d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:58.660392 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29e6bb-4cfd-44d5-95c9-c560ba030d9b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:58.697766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1004/2017] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:05:58.712509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0410d31-3a20-4875-bd6c-83b332bd6019 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:58.761271 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0410d31-3a20-4875-bd6c-83b332bd6019 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:58.787082 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220179e0-db73-4df6-bc8e-d1105c45e067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:58.797712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220179e0-db73-4df6-bc8e-d1105c45e067 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:58.804882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220179e0-db73-4df6-bc8e-d1105c45e067 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:58.804882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220179e0-db73-4df6-bc8e-d1105c45e067 admin admin] PATH_INFO: `/v3/projects/f53df0d106ab47a488fc636e8861f395/users/d88295fa9b634d07bcb3601d40738ccf/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:58.843975 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220179e0-db73-4df6-bc8e-d1105c45e067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f53df0d106ab47a488fc636e8861f395, user_id=d88295fa9b634d07bcb3601d40738ccf, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:58.850290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220179e0-db73-4df6-bc8e-d1105c45e067 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:58.854190 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:05:58.977563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1014/2018] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:05:58 2022] PUT /identity/v3/projects/f53df0d106ab47a488fc636e8861f395/users/d88295fa9b634d07bcb3601d40738ccf/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:05:58.996877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb52a0-dc8f-488a-ae71-42b5a0d8b5c1 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:58.997601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb52a0-dc8f-488a-ae71-42b5a0d8b5c1 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:58.997601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb52a0-dc8f-488a-ae71-42b5a0d8b5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:59.059975 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-18eb52a0-dc8f-488a-ae71-42b5a0d8b5c1 None None] MFA Rules not processed for user `d88295fa9b634d07bcb3601d40738ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:05:59.061694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-18eb52a0-dc8f-488a-ae71-42b5a0d8b5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:59.246260 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1005/2019] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:05:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:05:59.684696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220179e0-db73-4df6-bc8e-d1105c45e067 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:05:59.687913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-220179e0-db73-4df6-bc8e-d1105c45e067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:05:59.801547 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220179e0-db73-4df6-bc8e-d1105c45e067 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:05:59.805710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5dd1cc-fc2f-48a9-9f0c-f719ccf997bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:05:59.806289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dd1cc-fc2f-48a9-9f0c-f719ccf997bc service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:05:59.806620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dd1cc-fc2f-48a9-9f0c-f719ccf997bc service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:05:59.806908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dd1cc-fc2f-48a9-9f0c-f719ccf997bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:05:59.818892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5dd1cc-fc2f-48a9-9f0c-f719ccf997bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:05:59.820466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5dd1cc-fc2f-48a9-9f0c-f719ccf997bc service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:05:59.833678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1015/2020] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:05:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:06:09.212327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:INSERT=2,keystone:SELECT=39 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:09.828431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:26.732615 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb52a0-dc8f-488a-ae71-42b5a0d8b5c1 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:26.735035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-18eb52a0-dc8f-488a-ae71-42b5a0d8b5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:26.836657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb52a0-dc8f-488a-ae71-42b5a0d8b5c1 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:26.841473 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:26.842415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:26.842797 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:26.843051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] PATH_INFO: `/v3/users/8e16960c854543a8bdf141e278fac0c2` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:26.844225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e16960c854543a8bdf141e278fac0c2)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:26.846165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:27.010770 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e16960c854543a8bdf141e278fac0c2'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:27.011367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e16960c854543a8bdf141e278fac0c2'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:27.012051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e16960c854543a8bdf141e278fac0c2'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:27.105989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e16960c854543a8bdf141e278fac0c2'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:27.119778 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e16960c854543a8bdf141e278fac0c2'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:27.137366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1006/2021] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:06:26 2022] DELETE /identity/v3/users/8e16960c854543a8bdf141e278fac0c2 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:27.142772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7f4df-1765-4779-97cf-a1fb64a35545 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:27.143356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7f4df-1765-4779-97cf-a1fb64a35545 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:27.143356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7f4df-1765-4779-97cf-a1fb64a35545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:27.162792 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-c6d7f4df-1765-4779-97cf-a1fb64a35545 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:27.164089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c6d7f4df-1765-4779-97cf-a1fb64a35545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:27.230830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1016/2022] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:06:27.241420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:27.243081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:27.309132 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff3094-8fec-4889-99cd-7886b112a4e0 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:27.313418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17f562-f866-486e-a95e-e40eb192ca6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:27.314085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17f562-f866-486e-a95e-e40eb192ca6e service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:27.314399 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17f562-f866-486e-a95e-e40eb192ca6e service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:27.314683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17f562-f866-486e-a95e-e40eb192ca6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:27.324782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17f562-f866-486e-a95e-e40eb192ca6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:27.326264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17f562-f866-486e-a95e-e40eb192ca6e service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:27.341407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1007/2023] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:06:27.792081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7f4df-1765-4779-97cf-a1fb64a35545 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:27.793650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c6d7f4df-1765-4779-97cf-a1fb64a35545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:27.808830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7f4df-1765-4779-97cf-a1fb64a35545 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:27.814605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:27.815508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:27.815799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:27.816085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] PATH_INFO: `/v3/projects/cbde31b4a73b4dd59091f34c6d46dcb8` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:27.817653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbde31b4a73b4dd59091f34c6d46dcb8)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:27.819864 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:27.941873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbde31b4a73b4dd59091f34c6d46dcb8', 'request_id': 'req-78708067-c5d8-474b-be9b-c40260bb8a61'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:27.948784 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbde31b4a73b4dd59091f34c6d46dcb8', 'request_id': 'req-78708067-c5d8-474b-be9b-c40260bb8a61'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:27.955353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbde31b4a73b4dd59091f34c6d46dcb8', 'request_id': 'req-78708067-c5d8-474b-be9b-c40260bb8a61'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:28.042437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] The token cache is being invalidate because project cbde31b4a73b4dd59091f34c6d46dcb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:06:28.042716 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:28.043363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:28.044967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1017/2024] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:06:27 2022] DELETE /identity/v3/projects/cbde31b4a73b4dd59091f34c6d46dcb8 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:28.049310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17f562-f866-486e-a95e-e40eb192ca6e service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:28.059620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17f562-f866-486e-a95e-e40eb192ca6e service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:28.078621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:28.118796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d4d04-f905-4d6f-b458-92528843c514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:28.119723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d4d04-f905-4d6f-b458-92528843c514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:28.120010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d4d04-f905-4d6f-b458-92528843c514 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:28.120285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d4d04-f905-4d6f-b458-92528843c514 admin admin] PATH_INFO: `/v3/users/a2b2ff88a0db4d84a118f1f0c51868bf` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:28.121491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d4d04-f905-4d6f-b458-92528843c514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b2ff88a0db4d84a118f1f0c51868bf)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:28.123457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d4d04-f905-4d6f-b458-92528843c514 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:28.366590 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a17d4d04-f905-4d6f-b458-92528843c514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b2ff88a0db4d84a118f1f0c51868bf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:28.367335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a17d4d04-f905-4d6f-b458-92528843c514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b2ff88a0db4d84a118f1f0c51868bf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:28.367956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a17d4d04-f905-4d6f-b458-92528843c514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b2ff88a0db4d84a118f1f0c51868bf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:28.448656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a17d4d04-f905-4d6f-b458-92528843c514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b2ff88a0db4d84a118f1f0c51868bf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:28.463927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a17d4d04-f905-4d6f-b458-92528843c514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b2ff88a0db4d84a118f1f0c51868bf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:28.477277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1008/2025] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:06:28 2022] DELETE /identity/v3/users/a2b2ff88a0db4d84a118f1f0c51868bf => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:28.901826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:28.904604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:28.920541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78708067-c5d8-474b-be9b-c40260bb8a61 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:28.996931 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff420abb-55af-43dd-90e8-c66b95aa0b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:28.997812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff420abb-55af-43dd-90e8-c66b95aa0b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:28.998095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff420abb-55af-43dd-90e8-c66b95aa0b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:28.998409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff420abb-55af-43dd-90e8-c66b95aa0b7a admin admin] PATH_INFO: `/v3/projects/affa82ed045f4463b5f29a9dadfbbf06` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:28.999613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff420abb-55af-43dd-90e8-c66b95aa0b7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=affa82ed045f4463b5f29a9dadfbbf06)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:29.001785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff420abb-55af-43dd-90e8-c66b95aa0b7a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:29.118583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:29.135608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ff420abb-55af-43dd-90e8-c66b95aa0b7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'affa82ed045f4463b5f29a9dadfbbf06', 'request_id': 'req-ff420abb-55af-43dd-90e8-c66b95aa0b7a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:29.143022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ff420abb-55af-43dd-90e8-c66b95aa0b7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'affa82ed045f4463b5f29a9dadfbbf06', 'request_id': 'req-ff420abb-55af-43dd-90e8-c66b95aa0b7a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:29.150428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ff420abb-55af-43dd-90e8-c66b95aa0b7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'affa82ed045f4463b5f29a9dadfbbf06', 'request_id': 'req-ff420abb-55af-43dd-90e8-c66b95aa0b7a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:29.236285 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ff420abb-55af-43dd-90e8-c66b95aa0b7a admin admin] The token cache is being invalidate because project affa82ed045f4463b5f29a9dadfbbf06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:06:29.236581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ff420abb-55af-43dd-90e8-c66b95aa0b7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:29.237231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ff420abb-55af-43dd-90e8-c66b95aa0b7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:29.238994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1018/2026] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:06:28 2022] DELETE /identity/v3/projects/affa82ed045f4463b5f29a9dadfbbf06 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:29.244501 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1185b-aa24-4aa0-8d0e-80c6bfa5c974 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:29.245379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1185b-aa24-4aa0-8d0e-80c6bfa5c974 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:29.245665 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1185b-aa24-4aa0-8d0e-80c6bfa5c974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:29.266173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-94d1185b-aa24-4aa0-8d0e-80c6bfa5c974 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:29.267438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-94d1185b-aa24-4aa0-8d0e-80c6bfa5c974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:29.329433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1009/2027] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:29.337775 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0506c5-00e2-4b2d-9ea6-3229d7f11440 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:29.338066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0506c5-00e2-4b2d-9ea6-3229d7f11440 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:29.338397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0506c5-00e2-4b2d-9ea6-3229d7f11440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:29.364165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-3e0506c5-00e2-4b2d-9ea6-3229d7f11440 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:29.365527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3e0506c5-00e2-4b2d-9ea6-3229d7f11440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:29.375015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1019/2028] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:29.379281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1185b-aa24-4aa0-8d0e-80c6bfa5c974 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:29.389562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1185b-aa24-4aa0-8d0e-80c6bfa5c974 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:29.395210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f071e-f589-4174-97c9-bcb8af280bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:29.395887 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f071e-f589-4174-97c9-bcb8af280bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:29.397694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f071e-f589-4174-97c9-bcb8af280bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:29.397694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f071e-f589-4174-97c9-bcb8af280bc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:29.397694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f071e-f589-4174-97c9-bcb8af280bc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:06:29.397694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f071e-f589-4174-97c9-bcb8af280bc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:29.399127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f071e-f589-4174-97c9-bcb8af280bc1 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:29.412633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1010/2029] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:06:29.417401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0506c5-00e2-4b2d-9ea6-3229d7f11440 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:29.427910 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0506c5-00e2-4b2d-9ea6-3229d7f11440 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:29.432969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b0948-c938-4d30-a97f-edd80ddc705e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:29.433711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b0948-c938-4d30-a97f-edd80ddc705e admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:29.434006 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b0948-c938-4d30-a97f-edd80ddc705e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:29.434288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b0948-c938-4d30-a97f-edd80ddc705e admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:29.435052 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b0948-c938-4d30-a97f-edd80ddc705e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:29.437028 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b0948-c938-4d30-a97f-edd80ddc705e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:29.538059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1020/2030] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:06:29 2022] POST /identity/v3/projects => generated 369 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:29.542135 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f071e-f589-4174-97c9-bcb8af280bc1 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:29.553319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f071e-f589-4174-97c9-bcb8af280bc1 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:29.585144 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:29.635419 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dfd0b-13df-4d24-8086-e23c949470bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:29.636115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dfd0b-13df-4d24-8086-e23c949470bf admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:29.636405 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dfd0b-13df-4d24-8086-e23c949470bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:29.636690 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dfd0b-13df-4d24-8086-e23c949470bf admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:29.637397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38dfd0b-13df-4d24-8086-e23c949470bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:29.639371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38dfd0b-13df-4d24-8086-e23c949470bf admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:29.723851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1011/2031] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:06:29 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:29.728056 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b0948-c938-4d30-a97f-edd80ddc705e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:29.739088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b0948-c938-4d30-a97f-edd80ddc705e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:29.743874 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edcd84-418b-426e-9e5e-cce5da7ff796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:29.744546 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edcd84-418b-426e-9e5e-cce5da7ff796 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:29.744875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edcd84-418b-426e-9e5e-cce5da7ff796 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:29.745157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edcd84-418b-426e-9e5e-cce5da7ff796 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:29.745887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edcd84-418b-426e-9e5e-cce5da7ff796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:29.747513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edcd84-418b-426e-9e5e-cce5da7ff796 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:29.759710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1021/2032] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:06:29.764078 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dfd0b-13df-4d24-8086-e23c949470bf admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:29.775564 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dfd0b-13df-4d24-8086-e23c949470bf admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:29.780554 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffd17c-01e9-412f-bb91-a21b7c4c671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:29.781242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffd17c-01e9-412f-bb91-a21b7c4c671f admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:29.781532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffd17c-01e9-412f-bb91-a21b7c4c671f admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:29.781807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffd17c-01e9-412f-bb91-a21b7c4c671f admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:29.782625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffd17c-01e9-412f-bb91-a21b7c4c671f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:29.784224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffd17c-01e9-412f-bb91-a21b7c4c671f admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:29.796066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1012/2033] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:06:29.800191 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edcd84-418b-426e-9e5e-cce5da7ff796 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:29.813085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edcd84-418b-426e-9e5e-cce5da7ff796 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:29.819127 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0461e-67cf-412f-a1f4-ac5bf28ff519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:29.820123 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0461e-67cf-412f-a1f4-ac5bf28ff519 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:29.820400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0461e-67cf-412f-a1f4-ac5bf28ff519 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:29.820677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0461e-67cf-412f-a1f4-ac5bf28ff519 admin admin] PATH_INFO: `/v3/projects/fc4a6f32474b41d99ad6380df370c210/users/c740b820deef4b4195c03ceef5eeaac6/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:29.838071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0461e-67cf-412f-a1f4-ac5bf28ff519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4a6f32474b41d99ad6380df370c210, user_id=c740b820deef4b4195c03ceef5eeaac6, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:29.840866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0461e-67cf-412f-a1f4-ac5bf28ff519 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:29.912215 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1022/2034] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:06:29 2022] PUT /identity/v3/projects/fc4a6f32474b41d99ad6380df370c210/users/c740b820deef4b4195c03ceef5eeaac6/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:29.924835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4aa50-c0b4-48ae-be96-662df50359e7 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:29.925496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4aa50-c0b4-48ae-be96-662df50359e7 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:29.926064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4aa50-c0b4-48ae-be96-662df50359e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:29.971396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-36b4aa50-c0b4-48ae-be96-662df50359e7 None None] MFA Rules not processed for user `c740b820deef4b4195c03ceef5eeaac6`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:29.973122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-36b4aa50-c0b4-48ae-be96-662df50359e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:30.048423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1013/2035] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:30.058893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0461e-67cf-412f-a1f4-ac5bf28ff519 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:30.127193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0461e-67cf-412f-a1f4-ac5bf28ff519 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:30.132398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc20420-86ee-443a-8b49-fa8879f99e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:30.133395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc20420-86ee-443a-8b49-fa8879f99e12 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:30.133724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc20420-86ee-443a-8b49-fa8879f99e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:30.134013 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc20420-86ee-443a-8b49-fa8879f99e12 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:30.134760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc20420-86ee-443a-8b49-fa8879f99e12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:30.136714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc20420-86ee-443a-8b49-fa8879f99e12 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:30.234826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1023/2036] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:06:30 2022] POST /identity/v3/projects => generated 367 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:30.238935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4aa50-c0b4-48ae-be96-662df50359e7 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:30.250766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4aa50-c0b4-48ae-be96-662df50359e7 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:30.309371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab5082-b287-44c6-8d7e-b10c2378e87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:30.310057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab5082-b287-44c6-8d7e-b10c2378e87c admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:30.310377 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab5082-b287-44c6-8d7e-b10c2378e87c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:30.310666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab5082-b287-44c6-8d7e-b10c2378e87c admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:30.311370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab5082-b287-44c6-8d7e-b10c2378e87c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:30.313960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab5082-b287-44c6-8d7e-b10c2378e87c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:30.386714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1014/2037] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:06:30 2022] POST /identity/v3/users => generated 290 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:30.391468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc20420-86ee-443a-8b49-fa8879f99e12 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:30.403223 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc20420-86ee-443a-8b49-fa8879f99e12 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:30.408314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aeba47-384c-43d9-8f60-f01ff56a2069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:30.409018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeba47-384c-43d9-8f60-f01ff56a2069 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:30.409425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeba47-384c-43d9-8f60-f01ff56a2069 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:30.409718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeba47-384c-43d9-8f60-f01ff56a2069 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:30.410556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aeba47-384c-43d9-8f60-f01ff56a2069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:30.412216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aeba47-384c-43d9-8f60-f01ff56a2069 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:30.424416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1024/2038] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:06:30.428834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab5082-b287-44c6-8d7e-b10c2378e87c admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:30.440377 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab5082-b287-44c6-8d7e-b10c2378e87c admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:30.445208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4711323-d361-4940-bae7-8c4d8d05e13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:30.446099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4711323-d361-4940-bae7-8c4d8d05e13e admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:30.446488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4711323-d361-4940-bae7-8c4d8d05e13e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:30.446768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4711323-d361-4940-bae7-8c4d8d05e13e admin admin] PATH_INFO: `/v3/projects/0564eff0065745dc9aab108fb2e024e1/users/0c8f11a1e6e84eb290234333bdf53c43/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:30.461888 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4711323-d361-4940-bae7-8c4d8d05e13e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0564eff0065745dc9aab108fb2e024e1, user_id=0c8f11a1e6e84eb290234333bdf53c43, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:30.464622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4711323-d361-4940-bae7-8c4d8d05e13e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:30.539231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1015/2039] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:06:30 2022] PUT /identity/v3/projects/0564eff0065745dc9aab108fb2e024e1/users/0c8f11a1e6e84eb290234333bdf53c43/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:30.543226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aeba47-384c-43d9-8f60-f01ff56a2069 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:30.554283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aeba47-384c-43d9-8f60-f01ff56a2069 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:30.574389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:30.616272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f34d8-8a60-45f5-9452-4338bde9ef0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:30.616963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f34d8-8a60-45f5-9452-4338bde9ef0c admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:30.617319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f34d8-8a60-45f5-9452-4338bde9ef0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:30.617628 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f34d8-8a60-45f5-9452-4338bde9ef0c admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:30.618395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f34d8-8a60-45f5-9452-4338bde9ef0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:30.620021 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f34d8-8a60-45f5-9452-4338bde9ef0c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:30.630936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1025/2040] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:06:30.635639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4711323-d361-4940-bae7-8c4d8d05e13e admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:30.643769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:30.651008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4711323-d361-4940-bae7-8c4d8d05e13e admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:30.656591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b18c1-a95c-4bac-9d32-ab3a04506b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:30.657805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b18c1-a95c-4bac-9d32-ab3a04506b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:30.660150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b18c1-a95c-4bac-9d32-ab3a04506b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:30.660150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b18c1-a95c-4bac-9d32-ab3a04506b2a admin admin] PATH_INFO: `/v3/projects/0564eff0065745dc9aab108fb2e024e1/users/0c8f11a1e6e84eb290234333bdf53c43/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:30.660705 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b18c1-a95c-4bac-9d32-ab3a04506b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0564eff0065745dc9aab108fb2e024e1, user_id=0c8f11a1e6e84eb290234333bdf53c43, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:30.663394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b18c1-a95c-4bac-9d32-ab3a04506b2a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:30.677180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1016/2041] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:06:30 2022] PUT /identity/v3/projects/0564eff0065745dc9aab108fb2e024e1/users/0c8f11a1e6e84eb290234333bdf53c43/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:30.687783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e351f54-c310-4716-95f1-f67808bfe470 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:30.690166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e351f54-c310-4716-95f1-f67808bfe470 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:30.690166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e351f54-c310-4716-95f1-f67808bfe470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:30.719977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-7e351f54-c310-4716-95f1-f67808bfe470 None None] MFA Rules not processed for user `0c8f11a1e6e84eb290234333bdf53c43`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:30.723000 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7e351f54-c310-4716-95f1-f67808bfe470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:30.808592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1026/2042] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:30.820173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b18c1-a95c-4bac-9d32-ab3a04506b2a admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:30.821986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-fd5b18c1-a95c-4bac-9d32-ab3a04506b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:30.886589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b18c1-a95c-4bac-9d32-ab3a04506b2a admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:30.891032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d06ba-90e4-4739-a81a-21d00db2adef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:30.892032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d06ba-90e4-4739-a81a-21d00db2adef service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:30.892318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d06ba-90e4-4739-a81a-21d00db2adef service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:30.892602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d06ba-90e4-4739-a81a-21d00db2adef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:30.903002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d06ba-90e4-4739-a81a-21d00db2adef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:30.904507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d06ba-90e4-4739-a81a-21d00db2adef service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:30.917218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1017/2043] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:30 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:06:31.109238 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e351f54-c310-4716-95f1-f67808bfe470 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:31.125515 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e351f54-c310-4716-95f1-f67808bfe470 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:31.131714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef6af3-6c8e-47b6-898b-b1877977dda9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:31.133468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef6af3-6c8e-47b6-898b-b1877977dda9 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:31.133468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef6af3-6c8e-47b6-898b-b1877977dda9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:31.133954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef6af3-6c8e-47b6-898b-b1877977dda9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:31.218574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef6af3-6c8e-47b6-898b-b1877977dda9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:31.220272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef6af3-6c8e-47b6-898b-b1877977dda9 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:31.230189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1027/2044] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:06:35.839357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d06ba-90e4-4739-a81a-21d00db2adef service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:35.841024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-315d06ba-90e4-4739-a81a-21d00db2adef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:35.945623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d06ba-90e4-4739-a81a-21d00db2adef service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:35.951482 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:35.952722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:35.953032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:35.953323 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] PATH_INFO: `/v3/users/d88295fa9b634d07bcb3601d40738ccf` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:35.954669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d88295fa9b634d07bcb3601d40738ccf)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:35.956710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:36.234860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd88295fa9b634d07bcb3601d40738ccf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:36.235974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd88295fa9b634d07bcb3601d40738ccf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:36.236943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd88295fa9b634d07bcb3601d40738ccf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:36.345964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd88295fa9b634d07bcb3601d40738ccf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:36.361111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd88295fa9b634d07bcb3601d40738ccf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:36.377089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1018/2045] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:06:35 2022] DELETE /identity/v3/users/d88295fa9b634d07bcb3601d40738ccf => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:36.381798 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1d2c0-87e9-4ef2-9172-f511e0269d7a None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:36.382300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1d2c0-87e9-4ef2-9172-f511e0269d7a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:36.382430 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1d2c0-87e9-4ef2-9172-f511e0269d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:36.403893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-55a1d2c0-87e9-4ef2-9172-f511e0269d7a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:36.405187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-55a1d2c0-87e9-4ef2-9172-f511e0269d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:36.451811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:36.472215 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1028/2046] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:06:36.482440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:36.484593 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:36.501674 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:36.555399 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f2161-d081-4d5a-ae50-e5b660fd441c admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:36.559219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fba93-63ca-4739-bca9-92ca7434733a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:36.559891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fba93-63ca-4739-bca9-92ca7434733a service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:36.560182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fba93-63ca-4739-bca9-92ca7434733a service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:36.560472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fba93-63ca-4739-bca9-92ca7434733a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:36.570996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fba93-63ca-4739-bca9-92ca7434733a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:36.572492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fba93-63ca-4739-bca9-92ca7434733a service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:36.584764 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1019/2047] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:06:36.683871 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1d2c0-87e9-4ef2-9172-f511e0269d7a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:36.685313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-55a1d2c0-87e9-4ef2-9172-f511e0269d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:36.698353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1d2c0-87e9-4ef2-9172-f511e0269d7a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:36.703229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a0724-670e-456a-b918-8a98e074a034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:36.706115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a0724-670e-456a-b918-8a98e074a034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:36.706115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a0724-670e-456a-b918-8a98e074a034 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:36.706115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a0724-670e-456a-b918-8a98e074a034 admin admin] PATH_INFO: `/v3/users/c740b820deef4b4195c03ceef5eeaac6` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:36.706115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a0724-670e-456a-b918-8a98e074a034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c740b820deef4b4195c03ceef5eeaac6)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:36.708137 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a0724-670e-456a-b918-8a98e074a034 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:36.951111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d99a0724-670e-456a-b918-8a98e074a034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c740b820deef4b4195c03ceef5eeaac6'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:36.952783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d99a0724-670e-456a-b918-8a98e074a034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c740b820deef4b4195c03ceef5eeaac6'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:36.970096 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d99a0724-670e-456a-b918-8a98e074a034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c740b820deef4b4195c03ceef5eeaac6'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:36.983555 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fba93-63ca-4739-bca9-92ca7434733a service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:36.985295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-139fba93-63ca-4739-bca9-92ca7434733a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:36.988772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d99a0724-670e-456a-b918-8a98e074a034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c740b820deef4b4195c03ceef5eeaac6'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:37.001716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fba93-63ca-4739-bca9-92ca7434733a service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:37.007352 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51643e23-326b-4073-b6ca-8e1e0959ae77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:37.008533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51643e23-326b-4073-b6ca-8e1e0959ae77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:37.008819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51643e23-326b-4073-b6ca-8e1e0959ae77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:37.009098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51643e23-326b-4073-b6ca-8e1e0959ae77 admin admin] PATH_INFO: `/v3/users/d9f829be64744329ad991576e69512f5` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:37.010412 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51643e23-326b-4073-b6ca-8e1e0959ae77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9f829be64744329ad991576e69512f5)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:37.012547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51643e23-326b-4073-b6ca-8e1e0959ae77 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:37.146184 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d99a0724-670e-456a-b918-8a98e074a034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c740b820deef4b4195c03ceef5eeaac6'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:37.155108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1029/2048] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:06:36 2022] DELETE /identity/v3/users/c740b820deef4b4195c03ceef5eeaac6 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:37.163770 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035529f-18d2-4d54-8f38-ae01ed2899fe None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:37.164325 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035529f-18d2-4d54-8f38-ae01ed2899fe None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:37.164814 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035529f-18d2-4d54-8f38-ae01ed2899fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:37.194599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-6035529f-18d2-4d54-8f38-ae01ed2899fe None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:37.195859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6035529f-18d2-4d54-8f38-ae01ed2899fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:37.280751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1030/2049] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:06:37.291334 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035529f-18d2-4d54-8f38-ae01ed2899fe None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:37.293174 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6035529f-18d2-4d54-8f38-ae01ed2899fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:37.330735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-51643e23-326b-4073-b6ca-8e1e0959ae77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9f829be64744329ad991576e69512f5'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:37.331426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-51643e23-326b-4073-b6ca-8e1e0959ae77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9f829be64744329ad991576e69512f5'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:37.332107 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-51643e23-326b-4073-b6ca-8e1e0959ae77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9f829be64744329ad991576e69512f5'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:37.378085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035529f-18d2-4d54-8f38-ae01ed2899fe None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:37.381740 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb84130-e520-498f-9ae2-2f0885c6df4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:37.382451 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb84130-e520-498f-9ae2-2f0885c6df4e service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:37.383305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb84130-e520-498f-9ae2-2f0885c6df4e service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:37.383305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb84130-e520-498f-9ae2-2f0885c6df4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:37.385282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6cb84130-e520-498f-9ae2-2f0885c6df4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:37.397974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb84130-e520-498f-9ae2-2f0885c6df4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:37.399464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb84130-e520-498f-9ae2-2f0885c6df4e service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:37.404886 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:37.411851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1031/2050] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:06:37.425545 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-51643e23-326b-4073-b6ca-8e1e0959ae77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9f829be64744329ad991576e69512f5'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:37.445443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-51643e23-326b-4073-b6ca-8e1e0959ae77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9f829be64744329ad991576e69512f5'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:37.464332 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb84130-e520-498f-9ae2-2f0885c6df4e service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:37.464622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6cb84130-e520-498f-9ae2-2f0885c6df4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:37.482244 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1020/2051] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:06:36 2022] DELETE /identity/v3/users/d9f829be64744329ad991576e69512f5 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:06:37.492947 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4c876-39e6-48e2-9bad-cf2997c17a3f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:37.492947 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4c876-39e6-48e2-9bad-cf2997c17a3f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:37.492947 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4c876-39e6-48e2-9bad-cf2997c17a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:37.495286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb84130-e520-498f-9ae2-2f0885c6df4e service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:37.517844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-7ab4c876-39e6-48e2-9bad-cf2997c17a3f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:37.519703 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-7ab4c876-39e6-48e2-9bad-cf2997c17a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:37.597933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:37.598932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:37.599214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:37.599635 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] PATH_INFO: `/v3/projects/f53df0d106ab47a488fc636e8861f395` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:37.600948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f53df0d106ab47a488fc636e8861f395)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:37.602962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:37.608114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1021/2052] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:06:37.621758 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4c876-39e6-48e2-9bad-cf2997c17a3f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:37.623439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-7ab4c876-39e6-48e2-9bad-cf2997c17a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:37.664573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:37.704309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4c876-39e6-48e2-9bad-cf2997c17a3f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:37.709979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0c415-c523-4011-9852-081a7848c567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:37.710759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0c415-c523-4011-9852-081a7848c567 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:37.711199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0c415-c523-4011-9852-081a7848c567 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:37.711549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0c415-c523-4011-9852-081a7848c567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:37.725445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0c415-c523-4011-9852-081a7848c567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:37.727915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0c415-c523-4011-9852-081a7848c567 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:37.743108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1022/2053] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:06:37.790855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f53df0d106ab47a488fc636e8861f395', 'request_id': 'req-ced65c62-e3ae-4271-971c-42fad6a0fa45'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:37.799622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f53df0d106ab47a488fc636e8861f395', 'request_id': 'req-ced65c62-e3ae-4271-971c-42fad6a0fa45'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:37.815546 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f53df0d106ab47a488fc636e8861f395', 'request_id': 'req-ced65c62-e3ae-4271-971c-42fad6a0fa45'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:37.953673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] The token cache is being invalidate because project f53df0d106ab47a488fc636e8861f395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:06:37.954094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:37.954969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:37.957832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1032/2054] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:06:37 2022] DELETE /identity/v3/projects/f53df0d106ab47a488fc636e8861f395 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:06:37.962835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b210d-38a9-41a3-b073-9ae99e86c4f8 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:37.962936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b210d-38a9-41a3-b073-9ae99e86c4f8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:37.963221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b210d-38a9-41a3-b073-9ae99e86c4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:37.985735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-a08b210d-38a9-41a3-b073-9ae99e86c4f8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:37.987032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a08b210d-38a9-41a3-b073-9ae99e86c4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:38.061370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1023/2055] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:06:38.123512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:38.131806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:38.141562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635378d6-7341-4adc-bdc1-729c22772b91 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:38.141562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635378d6-7341-4adc-bdc1-729c22772b91 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:38.141562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635378d6-7341-4adc-bdc1-729c22772b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:38.171745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced65c62-e3ae-4271-971c-42fad6a0fa45 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:38.179356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-635378d6-7341-4adc-bdc1-729c22772b91 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:38.180632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-635378d6-7341-4adc-bdc1-729c22772b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:38.185888 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:38.189334 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:38.190008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:38.190671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] PATH_INFO: `/v3/projects/fc4a6f32474b41d99ad6380df370c210` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:38.192551 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc4a6f32474b41d99ad6380df370c210)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:38.194473 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1024/2056] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:38.195276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:38.195459 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635378d6-7341-4adc-bdc1-729c22772b91 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:38.208184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635378d6-7341-4adc-bdc1-729c22772b91 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:38.213868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83580e-1b78-44f6-a822-7616320d0107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:38.216175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83580e-1b78-44f6-a822-7616320d0107 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:38.216175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83580e-1b78-44f6-a822-7616320d0107 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:38.216175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83580e-1b78-44f6-a822-7616320d0107 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:38.216175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83580e-1b78-44f6-a822-7616320d0107 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:06:38.216175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83580e-1b78-44f6-a822-7616320d0107 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:38.218486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83580e-1b78-44f6-a822-7616320d0107 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:38.234123 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1025/2057] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:06:38.237995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83580e-1b78-44f6-a822-7616320d0107 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:38.249814 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83580e-1b78-44f6-a822-7616320d0107 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:38.255893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd239f-f126-46eb-94fa-5dde6b65a4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:38.257067 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd239f-f126-46eb-94fa-5dde6b65a4dc admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:38.257067 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd239f-f126-46eb-94fa-5dde6b65a4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:38.257255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd239f-f126-46eb-94fa-5dde6b65a4dc admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:38.258256 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd239f-f126-46eb-94fa-5dde6b65a4dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:38.260356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd239f-f126-46eb-94fa-5dde6b65a4dc admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:38.348328 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc4a6f32474b41d99ad6380df370c210', 'request_id': 'req-514e4012-f692-48fa-97ea-9820877d1c27'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:38.362759 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc4a6f32474b41d99ad6380df370c210', 'request_id': 'req-514e4012-f692-48fa-97ea-9820877d1c27'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:38.394232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc4a6f32474b41d99ad6380df370c210', 'request_id': 'req-514e4012-f692-48fa-97ea-9820877d1c27'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:38.426159 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1026/2058] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:06:38 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:06:38.431899 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd239f-f126-46eb-94fa-5dde6b65a4dc admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:38.516262 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] The token cache is being invalidate because project fc4a6f32474b41d99ad6380df370c210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:06:38.516633 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:38.518931 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:38.521641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1033/2059] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:06:38 2022] DELETE /identity/v3/projects/fc4a6f32474b41d99ad6380df370c210 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:38.525402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd239f-f126-46eb-94fa-5dde6b65a4dc admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:38.526015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:38.528166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:38.532635 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5dddf-082b-4d26-8fef-2365578ff5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:38.533410 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5dddf-082b-4d26-8fef-2365578ff5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:38.533788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5dddf-082b-4d26-8fef-2365578ff5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:38.534077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5dddf-082b-4d26-8fef-2365578ff5c8 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:38.534924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5dddf-082b-4d26-8fef-2365578ff5c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:38.537325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5dddf-082b-4d26-8fef-2365578ff5c8 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:38.547638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:38.606714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e4012-f692-48fa-97ea-9820877d1c27 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:38.617044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5fdd5-b61b-42d2-949b-230e0e1412e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:38.617044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5fdd5-b61b-42d2-949b-230e0e1412e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:38.617044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5fdd5-b61b-42d2-949b-230e0e1412e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:38.617044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5fdd5-b61b-42d2-949b-230e0e1412e5 admin admin] PATH_INFO: `/v3/users/0c8f11a1e6e84eb290234333bdf53c43` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:38.620171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5fdd5-b61b-42d2-949b-230e0e1412e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c8f11a1e6e84eb290234333bdf53c43)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:38.622530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5fdd5-b61b-42d2-949b-230e0e1412e5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:38.678724 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1027/2060] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:06:38 2022] POST /identity/v3/users => generated 292 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:06:38.679727 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5dddf-082b-4d26-8fef-2365578ff5c8 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:38.683274 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-fbb5dddf-082b-4d26-8fef-2365578ff5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:38.700204 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5dddf-082b-4d26-8fef-2365578ff5c8 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:38.710092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:38.711965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:38.713031 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:38.713486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] PATH_INFO: `/v3/projects/e0551a17b94a420186fe8f6d100b37f6` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:38.715396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0551a17b94a420186fe8f6d100b37f6)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:38.717011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:38.737520 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:38.905962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a9c5fdd5-b61b-42d2-949b-230e0e1412e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c8f11a1e6e84eb290234333bdf53c43'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:38.906621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0551a17b94a420186fe8f6d100b37f6', 'request_id': 'req-a5348038-575d-4a91-a430-2abf19826657'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:38.906749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a9c5fdd5-b61b-42d2-949b-230e0e1412e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c8f11a1e6e84eb290234333bdf53c43'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:38.914519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a9c5fdd5-b61b-42d2-949b-230e0e1412e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c8f11a1e6e84eb290234333bdf53c43'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:38.928550 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a9c5fdd5-b61b-42d2-949b-230e0e1412e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c8f11a1e6e84eb290234333bdf53c43'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:38.992321 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0551a17b94a420186fe8f6d100b37f6', 'request_id': 'req-a5348038-575d-4a91-a430-2abf19826657'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:39.002883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0551a17b94a420186fe8f6d100b37f6', 'request_id': 'req-a5348038-575d-4a91-a430-2abf19826657'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:39.021917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] The token cache is being invalidate because project e0551a17b94a420186fe8f6d100b37f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:06:39.022382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:39.023498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:39.025746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1028/2061] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:06:38 2022] DELETE /identity/v3/projects/e0551a17b94a420186fe8f6d100b37f6 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:39.028115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:39.032697 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:39.094860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a9c5fdd5-b61b-42d2-949b-230e0e1412e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c8f11a1e6e84eb290234333bdf53c43'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:39.102115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1034/2062] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:06:38 2022] DELETE /identity/v3/users/0c8f11a1e6e84eb290234333bdf53c43 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:06:39.103867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe2e51-8dce-4b93-9a7e-41156001de70 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.104165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe2e51-8dce-4b93-9a7e-41156001de70 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.104444 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe2e51-8dce-4b93-9a7e-41156001de70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.112734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5348038-575d-4a91-a430-2abf19826657 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:39.135397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-bafe2e51-8dce-4b93-9a7e-41156001de70 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:39.136621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-bafe2e51-8dce-4b93-9a7e-41156001de70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:39.197690 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd0b0f-9382-4464-93dc-db995a799908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:39.198815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0b0f-9382-4464-93dc-db995a799908 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.200203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0b0f-9382-4464-93dc-db995a799908 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.200662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0b0f-9382-4464-93dc-db995a799908 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.201554 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd0b0f-9382-4464-93dc-db995a799908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:39.203517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd0b0f-9382-4464-93dc-db995a799908 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:39.213732 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1035/2063] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:39.219957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1029/2064] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:06:39.222320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe2e51-8dce-4b93-9a7e-41156001de70 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:39.224110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-bafe2e51-8dce-4b93-9a7e-41156001de70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:39.224468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a9432-6c5b-4721-955f-876cfde4ccc8 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.224783 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a9432-6c5b-4721-955f-876cfde4ccc8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.225062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a9432-6c5b-4721-955f-876cfde4ccc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.239469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe2e51-8dce-4b93-9a7e-41156001de70 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:39.246122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fdc54-91b0-470b-8df0-af283dfbb56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:39.246671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fdc54-91b0-470b-8df0-af283dfbb56c admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.246960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fdc54-91b0-470b-8df0-af283dfbb56c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.247248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fdc54-91b0-470b-8df0-af283dfbb56c admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.248181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fdc54-91b0-470b-8df0-af283dfbb56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:39.248493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-828a9432-6c5b-4721-955f-876cfde4ccc8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:39.250513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fdc54-91b0-470b-8df0-af283dfbb56c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:39.250905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-828a9432-6c5b-4721-955f-876cfde4ccc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:39.262725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1036/2065] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:06:39.263610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1030/2066] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:39.267998 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a9432-6c5b-4721-955f-876cfde4ccc8 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:39.268323 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fdc54-91b0-470b-8df0-af283dfbb56c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:39.278266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a9432-6c5b-4721-955f-876cfde4ccc8 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:39.282671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fdc54-91b0-470b-8df0-af283dfbb56c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:39.287249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098b167-b037-44d6-8c26-fd0e02cb9fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:39.287249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098b167-b037-44d6-8c26-fd0e02cb9fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.287249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098b167-b037-44d6-8c26-fd0e02cb9fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.287249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098b167-b037-44d6-8c26-fd0e02cb9fa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.287648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098b167-b037-44d6-8c26-fd0e02cb9fa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:06:39.287648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098b167-b037-44d6-8c26-fd0e02cb9fa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:39.287957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098b167-b037-44d6-8c26-fd0e02cb9fa8 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:39.290192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6a379-8e55-4a85-b463-191e72a924ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:39.291245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6a379-8e55-4a85-b463-191e72a924ec admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.291531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6a379-8e55-4a85-b463-191e72a924ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.291807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6a379-8e55-4a85-b463-191e72a924ec admin admin] PATH_INFO: `/v3/projects/387ac04100224395a581aa382deb19b0/users/5321524c39c14135b5cc77e21598cbe1/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.307888 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1031/2067] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:06:39.317957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6a379-8e55-4a85-b463-191e72a924ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=387ac04100224395a581aa382deb19b0, user_id=5321524c39c14135b5cc77e21598cbe1, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:39.317957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6a379-8e55-4a85-b463-191e72a924ec admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:39.318813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098b167-b037-44d6-8c26-fd0e02cb9fa8 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:39.331335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098b167-b037-44d6-8c26-fd0e02cb9fa8 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:39.338695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40d6d6-c769-4d46-a0dd-e1d304f18aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:39.338695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40d6d6-c769-4d46-a0dd-e1d304f18aae admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.338695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40d6d6-c769-4d46-a0dd-e1d304f18aae admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.338695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40d6d6-c769-4d46-a0dd-e1d304f18aae admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.341470 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40d6d6-c769-4d46-a0dd-e1d304f18aae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:39.343831 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40d6d6-c769-4d46-a0dd-e1d304f18aae admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:39.434034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:39.444411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1037/2068] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:06:39 2022] PUT /identity/v3/projects/387ac04100224395a581aa382deb19b0/users/5321524c39c14135b5cc77e21598cbe1/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:39.451543 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b0880-92d6-4168-b6ec-b17ce43b7759 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.451843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b0880-92d6-4168-b6ec-b17ce43b7759 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.452170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b0880-92d6-4168-b6ec-b17ce43b7759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.477574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-634b0880-92d6-4168-b6ec-b17ce43b7759 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:39.479057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-634b0880-92d6-4168-b6ec-b17ce43b7759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:39.504490 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1032/2069] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:06:39 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:06:39.505532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40d6d6-c769-4d46-a0dd-e1d304f18aae admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:39.525895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:39.603771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1038/2070] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:39.611714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b0880-92d6-4168-b6ec-b17ce43b7759 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:39.615027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-634b0880-92d6-4168-b6ec-b17ce43b7759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:39.635253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40d6d6-c769-4d46-a0dd-e1d304f18aae admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:39.641043 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8ddeb-5be5-4144-ad4d-bbf7b77ac0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:39.641915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8ddeb-5be5-4144-ad4d-bbf7b77ac0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.646472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8ddeb-5be5-4144-ad4d-bbf7b77ac0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.646472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8ddeb-5be5-4144-ad4d-bbf7b77ac0b0 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.646472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8ddeb-5be5-4144-ad4d-bbf7b77ac0b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:39.646472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8ddeb-5be5-4144-ad4d-bbf7b77ac0b0 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:39.688814 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b0880-92d6-4168-b6ec-b17ce43b7759 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:39.770900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1033/2071] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:06:39 2022] POST /identity/v3/users => generated 287 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:39.772212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8ddeb-5be5-4144-ad4d-bbf7b77ac0b0 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:39.774897 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e03e4-d560-49f3-bf6f-1cf4a1cc467a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:39.775062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6ad8ddeb-5be5-4144-ad4d-bbf7b77ac0b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:39.775882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e03e4-d560-49f3-bf6f-1cf4a1cc467a service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.776483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e03e4-d560-49f3-bf6f-1cf4a1cc467a service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.776483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e03e4-d560-49f3-bf6f-1cf4a1cc467a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.787845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e03e4-d560-49f3-bf6f-1cf4a1cc467a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:39.790085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e03e4-d560-49f3-bf6f-1cf4a1cc467a service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:39.790259 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8ddeb-5be5-4144-ad4d-bbf7b77ac0b0 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:39.795207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4688eea-21c3-4304-a3fc-6875aaa1030a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:39.797007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4688eea-21c3-4304-a3fc-6875aaa1030a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.797401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4688eea-21c3-4304-a3fc-6875aaa1030a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.797686 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4688eea-21c3-4304-a3fc-6875aaa1030a admin admin] PATH_INFO: `/v3/projects/0564eff0065745dc9aab108fb2e024e1` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.798979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4688eea-21c3-4304-a3fc-6875aaa1030a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0564eff0065745dc9aab108fb2e024e1)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:39.801117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4688eea-21c3-4304-a3fc-6875aaa1030a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:39.805613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1039/2072] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:06:39.807321 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e03e4-d560-49f3-bf6f-1cf4a1cc467a service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:39.820636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e03e4-d560-49f3-bf6f-1cf4a1cc467a service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:39.828593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85673a3f-a920-4bc7-9ca6-314734bcfcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:39.829806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85673a3f-a920-4bc7-9ca6-314734bcfcd5 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.830395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85673a3f-a920-4bc7-9ca6-314734bcfcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.830996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85673a3f-a920-4bc7-9ca6-314734bcfcd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.836758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85673a3f-a920-4bc7-9ca6-314734bcfcd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:39.836758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85673a3f-a920-4bc7-9ca6-314734bcfcd5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:39.848734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1040/2073] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:06:39.852862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85673a3f-a920-4bc7-9ca6-314734bcfcd5 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:39.863709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85673a3f-a920-4bc7-9ca6-314734bcfcd5 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:39.868384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93a9c5-988a-4a8b-a335-e15a9948a4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:39.869049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93a9c5-988a-4a8b-a335-e15a9948a4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.869330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93a9c5-988a-4a8b-a335-e15a9948a4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.869607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93a9c5-988a-4a8b-a335-e15a9948a4a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.870348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93a9c5-988a-4a8b-a335-e15a9948a4a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:39.871938 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93a9c5-988a-4a8b-a335-e15a9948a4a5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:39.885520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1041/2074] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:06:39.890254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93a9c5-988a-4a8b-a335-e15a9948a4a5 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:39.900908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93a9c5-988a-4a8b-a335-e15a9948a4a5 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:39.906719 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3988e7-3588-4c47-b4e5-36fbdf1b80b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:39.907650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3988e7-3588-4c47-b4e5-36fbdf1b80b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:39.907937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3988e7-3588-4c47-b4e5-36fbdf1b80b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:39.908219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3988e7-3588-4c47-b4e5-36fbdf1b80b4 admin admin] PATH_INFO: `/v3/projects/c7fe45208d22480b88bd10c5382fed73/users/33615abbac2244f0a9050e17cf479776/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:39.923937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3988e7-3588-4c47-b4e5-36fbdf1b80b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7fe45208d22480b88bd10c5382fed73, user_id=33615abbac2244f0a9050e17cf479776, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:39.928520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3988e7-3588-4c47-b4e5-36fbdf1b80b4 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:40.026631 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1042/2075] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:06:39 2022] PUT /identity/v3/projects/c7fe45208d22480b88bd10c5382fed73/users/33615abbac2244f0a9050e17cf479776/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:40.038996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5c830-a192-4d57-adda-6ddf6711c15d None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:40.039295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5c830-a192-4d57-adda-6ddf6711c15d None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:40.039582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5c830-a192-4d57-adda-6ddf6711c15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:40.045200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d4688eea-21c3-4304-a3fc-6875aaa1030a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0564eff0065745dc9aab108fb2e024e1', 'request_id': 'req-d4688eea-21c3-4304-a3fc-6875aaa1030a'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:40.069180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-bbc5c830-a192-4d57-adda-6ddf6711c15d None None] MFA Rules not processed for user `33615abbac2244f0a9050e17cf479776`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:40.070521 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-bbc5c830-a192-4d57-adda-6ddf6711c15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:40.176554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1043/2076] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:06:40.185752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d4688eea-21c3-4304-a3fc-6875aaa1030a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0564eff0065745dc9aab108fb2e024e1', 'request_id': 'req-d4688eea-21c3-4304-a3fc-6875aaa1030a'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:40.191000 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5c830-a192-4d57-adda-6ddf6711c15d None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:40.199927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-bbc5c830-a192-4d57-adda-6ddf6711c15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:40.200083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d4688eea-21c3-4304-a3fc-6875aaa1030a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0564eff0065745dc9aab108fb2e024e1', 'request_id': 'req-d4688eea-21c3-4304-a3fc-6875aaa1030a'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:40.224237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d4688eea-21c3-4304-a3fc-6875aaa1030a admin admin] The token cache is being invalidate because project 0564eff0065745dc9aab108fb2e024e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:06:40.224815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d4688eea-21c3-4304-a3fc-6875aaa1030a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:40.225813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d4688eea-21c3-4304-a3fc-6875aaa1030a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:40.229137 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1034/2077] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:06:39 2022] DELETE /identity/v3/projects/0564eff0065745dc9aab108fb2e024e1 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:40.235025 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358b0a3-ec72-4460-9e35-6730278f6e9b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:40.235615 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358b0a3-ec72-4460-9e35-6730278f6e9b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:40.236082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358b0a3-ec72-4460-9e35-6730278f6e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:40.261205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e358b0a3-ec72-4460-9e35-6730278f6e9b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:40.262720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e358b0a3-ec72-4460-9e35-6730278f6e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:40.292272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5c830-a192-4d57-adda-6ddf6711c15d None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:40.296571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3047b9-d49d-4a6c-8672-522550cfdfcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:40.297254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3047b9-d49d-4a6c-8672-522550cfdfcc service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:40.297598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3047b9-d49d-4a6c-8672-522550cfdfcc service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:40.298102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3047b9-d49d-4a6c-8672-522550cfdfcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:40.300827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-4c3047b9-d49d-4a6c-8672-522550cfdfcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:40.313645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3047b9-d49d-4a6c-8672-522550cfdfcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:40.315178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3047b9-d49d-4a6c-8672-522550cfdfcc service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:40.324812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1044/2078] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:06:40.326152 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d519f4-f49b-4b84-8634-f1cc4de05840 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:40.326485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d519f4-f49b-4b84-8634-f1cc4de05840 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:40.326768 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d519f4-f49b-4b84-8634-f1cc4de05840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:40.336880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1035/2079] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:40.350111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96643ff-7e59-4f6f-8f67-e923d6fe2135 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:40.350111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96643ff-7e59-4f6f-8f67-e923d6fe2135 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:40.350111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96643ff-7e59-4f6f-8f67-e923d6fe2135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:40.354660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-27d519f4-f49b-4b84-8634-f1cc4de05840 None None] MFA Rules not processed for user `8d04f1378edc435a801a6c2719ec31f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:40.355998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-27d519f4-f49b-4b84-8634-f1cc4de05840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:40.383741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e96643ff-7e59-4f6f-8f67-e923d6fe2135 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:40.385064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e96643ff-7e59-4f6f-8f67-e923d6fe2135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:40.404368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1036/2080] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:06:40.409828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96643ff-7e59-4f6f-8f67-e923d6fe2135 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:40.431112 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96643ff-7e59-4f6f-8f67-e923d6fe2135 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:40.432180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627c741-2b7a-461f-837a-2225f022cafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:40.432890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627c741-2b7a-461f-837a-2225f022cafe admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:40.433234 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627c741-2b7a-461f-837a-2225f022cafe admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:40.433533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627c741-2b7a-461f-837a-2225f022cafe admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:40.434270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627c741-2b7a-461f-837a-2225f022cafe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:06:40.434633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627c741-2b7a-461f-837a-2225f022cafe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:40.438784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627c741-2b7a-461f-837a-2225f022cafe admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:40.447895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1045/2081] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:40.479995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1037/2082] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:06:40.481532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d519f4-f49b-4b84-8634-f1cc4de05840 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:40.497755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d519f4-f49b-4b84-8634-f1cc4de05840 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:40.505193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d6b37-d580-449a-92e8-d01f934cd826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:40.506007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d6b37-d580-449a-92e8-d01f934cd826 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:40.506447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d6b37-d580-449a-92e8-d01f934cd826 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:40.506852 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d6b37-d580-449a-92e8-d01f934cd826 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:40.507920 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d6b37-d580-449a-92e8-d01f934cd826 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:40.510171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d6b37-d580-449a-92e8-d01f934cd826 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:40.697169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1046/2083] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:06:40 2022] POST /identity/v3/projects => generated 371 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:40.701952 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627c741-2b7a-461f-837a-2225f022cafe admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:40.713724 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627c741-2b7a-461f-837a-2225f022cafe admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:40.788809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df898d7-54bc-49cb-a7a2-cbd184bc183e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:40.791061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df898d7-54bc-49cb-a7a2-cbd184bc183e admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:40.791499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df898d7-54bc-49cb-a7a2-cbd184bc183e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:40.792323 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df898d7-54bc-49cb-a7a2-cbd184bc183e admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:40.794716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df898d7-54bc-49cb-a7a2-cbd184bc183e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:40.794804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df898d7-54bc-49cb-a7a2-cbd184bc183e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:40.926364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1038/2084] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:06:40 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:40.928486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d6b37-d580-449a-92e8-d01f934cd826 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:40.930122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:40.941862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d6b37-d580-449a-92e8-d01f934cd826 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:40.946617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fcdef5-71b2-4289-b19f-8189ae404856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:40.954793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fcdef5-71b2-4289-b19f-8189ae404856 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:40.954793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fcdef5-71b2-4289-b19f-8189ae404856 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:40.954793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fcdef5-71b2-4289-b19f-8189ae404856 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:40.954793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fcdef5-71b2-4289-b19f-8189ae404856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:40.954793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fcdef5-71b2-4289-b19f-8189ae404856 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:40.962202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1047/2085] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:06:40.966575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df898d7-54bc-49cb-a7a2-cbd184bc183e admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:40.977939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df898d7-54bc-49cb-a7a2-cbd184bc183e admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:40.986129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715e47c-284e-4b3e-b555-5b82a6da49ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:40.986400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715e47c-284e-4b3e-b555-5b82a6da49ac admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:40.986597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715e47c-284e-4b3e-b555-5b82a6da49ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:40.990264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715e47c-284e-4b3e-b555-5b82a6da49ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:40.990264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715e47c-284e-4b3e-b555-5b82a6da49ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:40.990264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715e47c-284e-4b3e-b555-5b82a6da49ac admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:41.002921 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1039/2086] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:06:41.009469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fcdef5-71b2-4289-b19f-8189ae404856 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:41.020690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fcdef5-71b2-4289-b19f-8189ae404856 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:41.027288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe803e97-ba18-491c-a042-0deeb8275d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:41.028315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe803e97-ba18-491c-a042-0deeb8275d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:41.028610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe803e97-ba18-491c-a042-0deeb8275d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:41.028894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe803e97-ba18-491c-a042-0deeb8275d72 admin admin] PATH_INFO: `/v3/projects/02e0f85bb74149c0a452d19d33fe8194/users/6f07dd65478149209d8b8306e04ba980/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:41.052111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe803e97-ba18-491c-a042-0deeb8275d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e0f85bb74149c0a452d19d33fe8194, user_id=6f07dd65478149209d8b8306e04ba980, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:41.055005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe803e97-ba18-491c-a042-0deeb8275d72 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:41.146671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1048/2087] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:06:41 2022] PUT /identity/v3/projects/02e0f85bb74149c0a452d19d33fe8194/users/6f07dd65478149209d8b8306e04ba980/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:41.152045 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec821e1-ecba-41b9-be94-2342b548c1a1 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:41.152679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec821e1-ecba-41b9-be94-2342b548c1a1 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:41.152679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec821e1-ecba-41b9-be94-2342b548c1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:41.174589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-bec821e1-ecba-41b9-be94-2342b548c1a1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:41.176235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bec821e1-ecba-41b9-be94-2342b548c1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:41.249303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:41.288415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1040/2088] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:41.305025 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe803e97-ba18-491c-a042-0deeb8275d72 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:41.306867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fe803e97-ba18-491c-a042-0deeb8275d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:41.388585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe803e97-ba18-491c-a042-0deeb8275d72 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:41.395487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e6864-cfc1-44b2-bdc2-737eefc65022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:41.396489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e6864-cfc1-44b2-bdc2-737eefc65022 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:41.397420 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e6864-cfc1-44b2-bdc2-737eefc65022 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:41.399887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e6864-cfc1-44b2-bdc2-737eefc65022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:41.418429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e6864-cfc1-44b2-bdc2-737eefc65022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:41.420068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e6864-cfc1-44b2-bdc2-737eefc65022 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:41.426757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb14733-ccbc-40e4-8456-c29f295b622c None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:41.427015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb14733-ccbc-40e4-8456-c29f295b622c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:41.427318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb14733-ccbc-40e4-8456-c29f295b622c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:41.447506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1049/2089] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:06:41.463819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-8bb14733-ccbc-40e4-8456-c29f295b622c None None] MFA Rules not processed for user `5321524c39c14135b5cc77e21598cbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:41.465465 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8bb14733-ccbc-40e4-8456-c29f295b622c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:41.568296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1041/2090] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:41.583221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e6864-cfc1-44b2-bdc2-737eefc65022 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:41.584962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-910e6864-cfc1-44b2-bdc2-737eefc65022 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:41.684415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e6864-cfc1-44b2-bdc2-737eefc65022 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:41.690668 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0bb76-f5d7-4d80-aba2-816c16803b1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:41.694054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0bb76-f5d7-4d80-aba2-816c16803b1b service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:41.694054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0bb76-f5d7-4d80-aba2-816c16803b1b service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:41.694054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0bb76-f5d7-4d80-aba2-816c16803b1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:41.704348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0bb76-f5d7-4d80-aba2-816c16803b1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:41.716778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0bb76-f5d7-4d80-aba2-816c16803b1b service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:41.716778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1050/2091] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:06:42.026276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:43.542244 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d86a5-41d8-4acd-aa64-650a9d43ae19 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:43.542820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d86a5-41d8-4acd-aa64-650a9d43ae19 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:43.543192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d86a5-41d8-4acd-aa64-650a9d43ae19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:43.554366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:43.643599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-281d86a5-41d8-4acd-aa64-650a9d43ae19 None None] MFA Rules not processed for user `6f07dd65478149209d8b8306e04ba980`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:43.644863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-281d86a5-41d8-4acd-aa64-650a9d43ae19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:43.766221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1042/2092] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:43.772976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0bb76-f5d7-4d80-aba2-816c16803b1b service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:43.793646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0bb76-f5d7-4d80-aba2-816c16803b1b service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:43.800167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81338b07-b78c-40df-977d-0de67fd7ebd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:43.800949 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81338b07-b78c-40df-977d-0de67fd7ebd1 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:43.801436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81338b07-b78c-40df-977d-0de67fd7ebd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:43.801865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81338b07-b78c-40df-977d-0de67fd7ebd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:43.802790 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81338b07-b78c-40df-977d-0de67fd7ebd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:43.804721 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81338b07-b78c-40df-977d-0de67fd7ebd1 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:43.966803 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1051/2093] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:06:43 2022] POST /identity/v3/projects => generated 373 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:43.972437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d86a5-41d8-4acd-aa64-650a9d43ae19 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:43.987943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d86a5-41d8-4acd-aa64-650a9d43ae19 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:44.205413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b05136-21b7-4f95-bf2d-a29f8d9a64d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:44.206357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b05136-21b7-4f95-bf2d-a29f8d9a64d6 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:44.206844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b05136-21b7-4f95-bf2d-a29f8d9a64d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:44.207140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b05136-21b7-4f95-bf2d-a29f8d9a64d6 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:44.207996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b05136-21b7-4f95-bf2d-a29f8d9a64d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:44.210457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b05136-21b7-4f95-bf2d-a29f8d9a64d6 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:44.331410 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1043/2094] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:06:43 2022] POST /identity/v3/users => generated 293 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:44.334135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81338b07-b78c-40df-977d-0de67fd7ebd1 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:44.348058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81338b07-b78c-40df-977d-0de67fd7ebd1 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:44.353477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65523302-ef62-4d14-ae08-a6d4da7e73af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:44.354167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65523302-ef62-4d14-ae08-a6d4da7e73af admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:44.354488 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65523302-ef62-4d14-ae08-a6d4da7e73af admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:44.359809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65523302-ef62-4d14-ae08-a6d4da7e73af admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:44.359809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65523302-ef62-4d14-ae08-a6d4da7e73af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:44.359809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65523302-ef62-4d14-ae08-a6d4da7e73af admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:44.370879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1052/2095] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:06:44.376135 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b05136-21b7-4f95-bf2d-a29f8d9a64d6 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:44.387845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b05136-21b7-4f95-bf2d-a29f8d9a64d6 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:44.394701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ddc38-1252-419b-a9b7-eef91159e64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:44.395680 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ddc38-1252-419b-a9b7-eef91159e64f admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:44.400032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ddc38-1252-419b-a9b7-eef91159e64f admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:44.400032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ddc38-1252-419b-a9b7-eef91159e64f admin admin] PATH_INFO: `/v3/projects/510a2d4c29044244b973a54e712b31f9/users/718447055e6f412397b8c7c7a2f51122/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:44.415996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ddc38-1252-419b-a9b7-eef91159e64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=510a2d4c29044244b973a54e712b31f9, user_id=718447055e6f412397b8c7c7a2f51122, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:44.419516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ddc38-1252-419b-a9b7-eef91159e64f admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:44.518435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1044/2096] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:06:44 2022] PUT /identity/v3/projects/510a2d4c29044244b973a54e712b31f9/users/718447055e6f412397b8c7c7a2f51122/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:06:44.522964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65523302-ef62-4d14-ae08-a6d4da7e73af admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:44.534830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65523302-ef62-4d14-ae08-a6d4da7e73af admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:44.605736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1398f-5f99-414d-ab78-496246995daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:44.606499 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1398f-5f99-414d-ab78-496246995daf admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:44.606843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1398f-5f99-414d-ab78-496246995daf admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:44.607158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1398f-5f99-414d-ab78-496246995daf admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:44.607929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1398f-5f99-414d-ab78-496246995daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:44.610275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1398f-5f99-414d-ab78-496246995daf admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:44.618083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:44.630313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1053/2097] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:06:44.631632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ddc38-1252-419b-a9b7-eef91159e64f admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:44.661522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ddc38-1252-419b-a9b7-eef91159e64f admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:44.678793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9f800-4b65-475d-a481-5c2bcb9b0e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:44.680359 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9f800-4b65-475d-a481-5c2bcb9b0e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:44.680559 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9f800-4b65-475d-a481-5c2bcb9b0e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:44.687177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9f800-4b65-475d-a481-5c2bcb9b0e15 admin admin] PATH_INFO: `/v3/projects/510a2d4c29044244b973a54e712b31f9/users/718447055e6f412397b8c7c7a2f51122/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:44.687177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9f800-4b65-475d-a481-5c2bcb9b0e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510a2d4c29044244b973a54e712b31f9, user_id=718447055e6f412397b8c7c7a2f51122, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:44.689129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9f800-4b65-475d-a481-5c2bcb9b0e15 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:44.731542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1045/2098] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:06:44 2022] PUT /identity/v3/projects/510a2d4c29044244b973a54e712b31f9/users/718447055e6f412397b8c7c7a2f51122/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:06:45.869144 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df48f0-9221-409d-bf28-1623250054f7 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:45.870454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df48f0-9221-409d-bf28-1623250054f7 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:45.870823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df48f0-9221-409d-bf28-1623250054f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:45.894621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-54df48f0-9221-409d-bf28-1623250054f7 None None] MFA Rules not processed for user `5321524c39c14135b5cc77e21598cbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:45.896010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-54df48f0-9221-409d-bf28-1623250054f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:45.971698 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1054/2099] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:46.475462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9f800-4b65-475d-a481-5c2bcb9b0e15 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:46.478200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f8e9f800-4b65-475d-a481-5c2bcb9b0e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:46.559982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:46.579122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9f800-4b65-475d-a481-5c2bcb9b0e15 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:46.585300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:46.586450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:46.586924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:46.587310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] PATH_INFO: `/v3/users/33615abbac2244f0a9050e17cf479776` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:46.588956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33615abbac2244f0a9050e17cf479776)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:46.591736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:46.855241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33615abbac2244f0a9050e17cf479776'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:46.857483 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33615abbac2244f0a9050e17cf479776'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:46.861212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33615abbac2244f0a9050e17cf479776'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:46.967173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33615abbac2244f0a9050e17cf479776'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:46.996745 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33615abbac2244f0a9050e17cf479776'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:47.020704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1046/2100] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:06:46 2022] DELETE /identity/v3/users/33615abbac2244f0a9050e17cf479776 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:47.022147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fee3a7-c124-434f-a607-9a2a19bd3d32 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:47.022998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fee3a7-c124-434f-a607-9a2a19bd3d32 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:47.022998 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fee3a7-c124-434f-a607-9a2a19bd3d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:47.058655 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-28fee3a7-c124-434f-a607-9a2a19bd3d32 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:47.059966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-28fee3a7-c124-434f-a607-9a2a19bd3d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:47.158595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1055/2101] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:47.166505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:47.168067 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:47.238903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4d947-0bc2-4de5-96e2-d98f7d14ce12 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:47.243036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d70ca9-bf6a-4f5f-8b96-94116760ec85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:47.243529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d70ca9-bf6a-4f5f-8b96-94116760ec85 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:47.243815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d70ca9-bf6a-4f5f-8b96-94116760ec85 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:47.244089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d70ca9-bf6a-4f5f-8b96-94116760ec85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:47.254847 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d70ca9-bf6a-4f5f-8b96-94116760ec85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:47.256357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d70ca9-bf6a-4f5f-8b96-94116760ec85 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:47.276682 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1047/2102] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:06:47.545281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f075b3-4936-4392-a49c-8576b3938bd4 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:47.545281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f075b3-4936-4392-a49c-8576b3938bd4 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:47.546019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f075b3-4936-4392-a49c-8576b3938bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:47.580682 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-04f075b3-4936-4392-a49c-8576b3938bd4 None None] MFA Rules not processed for user `718447055e6f412397b8c7c7a2f51122`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:47.582179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-04f075b3-4936-4392-a49c-8576b3938bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:47.782471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1056/2103] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:47.797330 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d70ca9-bf6a-4f5f-8b96-94116760ec85 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:47.801242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c3d70ca9-bf6a-4f5f-8b96-94116760ec85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:48.024024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d70ca9-bf6a-4f5f-8b96-94116760ec85 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:48.036858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808a461-0c43-4e81-8660-b18716b74793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:48.037552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808a461-0c43-4e81-8660-b18716b74793 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:48.037836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808a461-0c43-4e81-8660-b18716b74793 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:48.038293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808a461-0c43-4e81-8660-b18716b74793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:48.047424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6808a461-0c43-4e81-8660-b18716b74793 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:48.118887 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:48.237695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808a461-0c43-4e81-8660-b18716b74793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:48.241125 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808a461-0c43-4e81-8660-b18716b74793 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:48.253024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1048/2104] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:06:48.285333 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f075b3-4936-4392-a49c-8576b3938bd4 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:48.287365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-04f075b3-4936-4392-a49c-8576b3938bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:48.318737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f075b3-4936-4392-a49c-8576b3938bd4 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:48.351052 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f0998-c3eb-40cf-bb35-5995c97025ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:48.352289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f0998-c3eb-40cf-bb35-5995c97025ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:48.353209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f0998-c3eb-40cf-bb35-5995c97025ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:48.353717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f0998-c3eb-40cf-bb35-5995c97025ca admin admin] PATH_INFO: `/v3/projects/c7fe45208d22480b88bd10c5382fed73` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:48.364125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f0998-c3eb-40cf-bb35-5995c97025ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7fe45208d22480b88bd10c5382fed73)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:48.368107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f0998-c3eb-40cf-bb35-5995c97025ca admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:48.550237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:48.584921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e56f0998-c3eb-40cf-bb35-5995c97025ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7fe45208d22480b88bd10c5382fed73', 'request_id': 'req-e56f0998-c3eb-40cf-bb35-5995c97025ca'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:48.593367 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e56f0998-c3eb-40cf-bb35-5995c97025ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7fe45208d22480b88bd10c5382fed73', 'request_id': 'req-e56f0998-c3eb-40cf-bb35-5995c97025ca'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:48.600727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e56f0998-c3eb-40cf-bb35-5995c97025ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7fe45208d22480b88bd10c5382fed73', 'request_id': 'req-e56f0998-c3eb-40cf-bb35-5995c97025ca'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:48.683212 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e56f0998-c3eb-40cf-bb35-5995c97025ca admin admin] The token cache is being invalidate because project c7fe45208d22480b88bd10c5382fed73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:06:48.684016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e56f0998-c3eb-40cf-bb35-5995c97025ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:48.685817 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e56f0998-c3eb-40cf-bb35-5995c97025ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:48.690984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1057/2105] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:06:48 2022] DELETE /identity/v3/projects/c7fe45208d22480b88bd10c5382fed73 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:48.697561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57731fe7-656d-475b-82da-8f09dae5f801 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:48.698358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57731fe7-656d-475b-82da-8f09dae5f801 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:48.698916 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57731fe7-656d-475b-82da-8f09dae5f801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:48.738754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-57731fe7-656d-475b-82da-8f09dae5f801 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:48.740499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-57731fe7-656d-475b-82da-8f09dae5f801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:48.827332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1049/2106] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:48.837323 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a2c90-fd3a-4589-a7e3-8496a632c70f None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:48.838008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a2c90-fd3a-4589-a7e3-8496a632c70f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:48.838310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a2c90-fd3a-4589-a7e3-8496a632c70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:48.867836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-9e3a2c90-fd3a-4589-a7e3-8496a632c70f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:48.869243 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9e3a2c90-fd3a-4589-a7e3-8496a632c70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:48.885781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1058/2107] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:48.886737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57731fe7-656d-475b-82da-8f09dae5f801 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:48.912272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57731fe7-656d-475b-82da-8f09dae5f801 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:48.917732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f812a-f6f0-44c5-a1aa-dcbc045f222b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:48.918453 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f812a-f6f0-44c5-a1aa-dcbc045f222b admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:48.918948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f812a-f6f0-44c5-a1aa-dcbc045f222b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:48.919231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f812a-f6f0-44c5-a1aa-dcbc045f222b admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:48.919976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f812a-f6f0-44c5-a1aa-dcbc045f222b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:06:48.920573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f812a-f6f0-44c5-a1aa-dcbc045f222b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:48.923091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f812a-f6f0-44c5-a1aa-dcbc045f222b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:48.938232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1050/2108] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:06:48.943293 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a2c90-fd3a-4589-a7e3-8496a632c70f None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:48.954739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a2c90-fd3a-4589-a7e3-8496a632c70f None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:48.960832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89496b41-255b-46e2-b6d2-0cdac6eeaaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:48.961930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89496b41-255b-46e2-b6d2-0cdac6eeaaff admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:48.962425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89496b41-255b-46e2-b6d2-0cdac6eeaaff admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:48.962928 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89496b41-255b-46e2-b6d2-0cdac6eeaaff admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:48.963819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89496b41-255b-46e2-b6d2-0cdac6eeaaff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:48.965741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89496b41-255b-46e2-b6d2-0cdac6eeaaff admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:49.072044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1059/2109] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:06:48 2022] POST /identity/v3/projects => generated 391 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:49.076332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f812a-f6f0-44c5-a1aa-dcbc045f222b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.091394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f812a-f6f0-44c5-a1aa-dcbc045f222b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.185777 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cf0dc-b93b-4eb7-9431-178e4d076edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:49.186797 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf0dc-b93b-4eb7-9431-178e4d076edd admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.187522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf0dc-b93b-4eb7-9431-178e4d076edd admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.187967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf0dc-b93b-4eb7-9431-178e4d076edd admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.188871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cf0dc-b93b-4eb7-9431-178e4d076edd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:49.191698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cf0dc-b93b-4eb7-9431-178e4d076edd admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:49.332735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1051/2110] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:06:49 2022] POST /identity/v3/users => generated 296 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:06:49.336594 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89496b41-255b-46e2-b6d2-0cdac6eeaaff admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.349150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89496b41-255b-46e2-b6d2-0cdac6eeaaff admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.359238 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca5f0b-7ec8-4a75-aa58-6a1ce52fe935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:49.360018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca5f0b-7ec8-4a75-aa58-6a1ce52fe935 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.360649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca5f0b-7ec8-4a75-aa58-6a1ce52fe935 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.361180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca5f0b-7ec8-4a75-aa58-6a1ce52fe935 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.362356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca5f0b-7ec8-4a75-aa58-6a1ce52fe935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:49.365520 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca5f0b-7ec8-4a75-aa58-6a1ce52fe935 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:49.382532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1060/2111] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:06:49.386031 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cf0dc-b93b-4eb7-9431-178e4d076edd admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.398787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cf0dc-b93b-4eb7-9431-178e4d076edd admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.404124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7b594-b870-4955-9589-f26b3f093d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:49.404952 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7b594-b870-4955-9589-f26b3f093d39 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.405157 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7b594-b870-4955-9589-f26b3f093d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.405607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7b594-b870-4955-9589-f26b3f093d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.406239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7b594-b870-4955-9589-f26b3f093d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:49.407968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7b594-b870-4955-9589-f26b3f093d39 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:49.423451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1052/2112] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:06:49.427783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca5f0b-7ec8-4a75-aa58-6a1ce52fe935 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.443567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca5f0b-7ec8-4a75-aa58-6a1ce52fe935 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.449360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f88b6-9a01-4db3-91e7-580f0523868c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:49.450283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f88b6-9a01-4db3-91e7-580f0523868c admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.450615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f88b6-9a01-4db3-91e7-580f0523868c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.450892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f88b6-9a01-4db3-91e7-580f0523868c admin admin] PATH_INFO: `/v3/projects/b17fe8a7474e4fbfa0bd16e5e879ef96/users/3ed7496968364068b430c3d878da3f45/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.470489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f88b6-9a01-4db3-91e7-580f0523868c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b17fe8a7474e4fbfa0bd16e5e879ef96, user_id=3ed7496968364068b430c3d878da3f45, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:49.477845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f88b6-9a01-4db3-91e7-580f0523868c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:49.508893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc517f25-4342-45cd-b1bc-53ace081b4ac None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.509594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc517f25-4342-45cd-b1bc-53ace081b4ac None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.509890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc517f25-4342-45cd-b1bc-53ace081b4ac None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.511633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1053/2113] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 19:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:06:49.544435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a59c87-9cad-4888-be66-ad8b7bb05bdd None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.544828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a59c87-9cad-4888-be66-ad8b7bb05bdd None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.545186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a59c87-9cad-4888-be66-ad8b7bb05bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.571246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-63a59c87-9cad-4888-be66-ad8b7bb05bdd None None] MFA Rules not processed for user `f5027a60da214afda9144b5ae6cce15b`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:49.572571 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-63a59c87-9cad-4888-be66-ad8b7bb05bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:49.604389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1061/2114] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:06:49 2022] PUT /identity/v3/projects/b17fe8a7474e4fbfa0bd16e5e879ef96/users/3ed7496968364068b430c3d878da3f45/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:49.617458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96a774-4d6e-4b0c-92ba-8e4530ef1e64 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.617721 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96a774-4d6e-4b0c-92ba-8e4530ef1e64 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.617896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96a774-4d6e-4b0c-92ba-8e4530ef1e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.665600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-da96a774-4d6e-4b0c-92ba-8e4530ef1e64 None None] MFA Rules not processed for user `3ed7496968364068b430c3d878da3f45`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:49.667029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-da96a774-4d6e-4b0c-92ba-8e4530ef1e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:49.677352 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1054/2115] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 19:06:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:06:49.694306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a59c87-9cad-4888-be66-ad8b7bb05bdd None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.696413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-63a59c87-9cad-4888-be66-ad8b7bb05bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:49.711432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a59c87-9cad-4888-be66-ad8b7bb05bdd None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.734776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505ce23-1e8b-4240-a138-bb259a7b3b5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:49.735979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505ce23-1e8b-4240-a138-bb259a7b3b5a service placement] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.737032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505ce23-1e8b-4240-a138-bb259a7b3b5a service placement] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.737032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505ce23-1e8b-4240-a138-bb259a7b3b5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.749749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505ce23-1e8b-4240-a138-bb259a7b3b5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:49.751867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505ce23-1e8b-4240-a138-bb259a7b3b5a service placement] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:49.768024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1055/2116] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:06:49.768589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505ce23-1e8b-4240-a138-bb259a7b3b5a service placement] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.770102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d505ce23-1e8b-4240-a138-bb259a7b3b5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:49.783471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1062/2117] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:06:49.787181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:49.797323 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96a774-4d6e-4b0c-92ba-8e4530ef1e64 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.800594 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-da96a774-4d6e-4b0c-92ba-8e4530ef1e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:49.867734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:49.871883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505ce23-1e8b-4240-a138-bb259a7b3b5a service placement] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.879099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03da4a-ab64-459f-acba-2474a38c9eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:49.880214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03da4a-ab64-459f-acba-2474a38c9eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.880702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03da4a-ab64-459f-acba-2474a38c9eef admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.881184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03da4a-ab64-459f-acba-2474a38c9eef admin admin] PATH_INFO: `/v3/users/c0464996da4f44008ca8b91c837ac22d` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.883237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03da4a-ab64-459f-acba-2474a38c9eef admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0464996da4f44008ca8b91c837ac22d)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:49.886688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03da4a-ab64-459f-acba-2474a38c9eef admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:49.889850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96a774-4d6e-4b0c-92ba-8e4530ef1e64 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.895948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38efde1-4159-441d-ace5-5dd49f17c1d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:49.898266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38efde1-4159-441d-ace5-5dd49f17c1d2 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.898266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38efde1-4159-441d-ace5-5dd49f17c1d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.898266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38efde1-4159-441d-ace5-5dd49f17c1d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.915881 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38efde1-4159-441d-ace5-5dd49f17c1d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:49.917547 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38efde1-4159-441d-ace5-5dd49f17c1d2 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:49.929549 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1063/2118] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:06:50.168141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6e03da4a-ab64-459f-acba-2474a38c9eef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0464996da4f44008ca8b91c837ac22d'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:50.168666 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6e03da4a-ab64-459f-acba-2474a38c9eef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0464996da4f44008ca8b91c837ac22d'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:50.169380 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6e03da4a-ab64-459f-acba-2474a38c9eef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0464996da4f44008ca8b91c837ac22d'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:50.293016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6e03da4a-ab64-459f-acba-2474a38c9eef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0464996da4f44008ca8b91c837ac22d'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:50.308917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6e03da4a-ab64-459f-acba-2474a38c9eef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0464996da4f44008ca8b91c837ac22d'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:50.323804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1056/2119] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:06:49 2022] DELETE /identity/v3/users/c0464996da4f44008ca8b91c837ac22d => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:50.663205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38efde1-4159-441d-ace5-5dd49f17c1d2 service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:50.687253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38efde1-4159-441d-ace5-5dd49f17c1d2 service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:50.774493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:50.779620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:50.782802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:50.782802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1 admin admin] PATH_INFO: `/v3/projects/463f573db9d84c03900d443e4f6585cd` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:50.782802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=463f573db9d84c03900d443e4f6585cd)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:50.784652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:50.902636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '463f573db9d84c03900d443e4f6585cd', 'request_id': 'req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:50.909372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '463f573db9d84c03900d443e4f6585cd', 'request_id': 'req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:50.915917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '463f573db9d84c03900d443e4f6585cd', 'request_id': 'req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:51.031179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1 admin admin] The token cache is being invalidate because project 463f573db9d84c03900d443e4f6585cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:06:51.031505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:51.032190 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-91a62ccf-523d-464f-86cf-bd7f0af1ebd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:06:51.033769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1064/2120] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:06:50 2022] DELETE /identity/v3/projects/463f573db9d84c03900d443e4f6585cd => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:51.039073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638fe4-43f1-401f-99cf-32373a6c9756 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:51.039621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638fe4-43f1-401f-99cf-32373a6c9756 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:51.039748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638fe4-43f1-401f-99cf-32373a6c9756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:51.060495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c2638fe4-43f1-401f-99cf-32373a6c9756 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:51.062440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c2638fe4-43f1-401f-99cf-32373a6c9756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:51.096401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:51.138901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1057/2121] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:51.147405 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3ee28-0a30-4ea8-bed4-a2a7a75e8579 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:51.147609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3ee28-0a30-4ea8-bed4-a2a7a75e8579 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:51.147840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3ee28-0a30-4ea8-bed4-a2a7a75e8579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:51.169789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-6cd3ee28-0a30-4ea8-bed4-a2a7a75e8579 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:51.171606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6cd3ee28-0a30-4ea8-bed4-a2a7a75e8579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:51.180600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1065/2122] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:51.185147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2638fe4-43f1-401f-99cf-32373a6c9756 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:51.196235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2638fe4-43f1-401f-99cf-32373a6c9756 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:51.201084 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e77c9-3428-45cb-a4be-7929a7e47a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:51.201857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e77c9-3428-45cb-a4be-7929a7e47a58 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:51.202039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e77c9-3428-45cb-a4be-7929a7e47a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:51.202384 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e77c9-3428-45cb-a4be-7929a7e47a58 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:51.203281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e77c9-3428-45cb-a4be-7929a7e47a58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:06:51.203593 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e77c9-3428-45cb-a4be-7929a7e47a58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:51.205240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e77c9-3428-45cb-a4be-7929a7e47a58 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:51.218464 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1058/2123] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:06:51.227583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3ee28-0a30-4ea8-bed4-a2a7a75e8579 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:51.238425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3ee28-0a30-4ea8-bed4-a2a7a75e8579 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:51.243749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904853e4-2cff-4126-b7d4-d7efcbd1998e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:51.244380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904853e4-2cff-4126-b7d4-d7efcbd1998e admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:51.244659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904853e4-2cff-4126-b7d4-d7efcbd1998e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:51.244993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904853e4-2cff-4126-b7d4-d7efcbd1998e admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:51.245955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904853e4-2cff-4126-b7d4-d7efcbd1998e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:51.248313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904853e4-2cff-4126-b7d4-d7efcbd1998e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:51.352834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1066/2124] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:06:51 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:51.357359 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e77c9-3428-45cb-a4be-7929a7e47a58 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:51.376868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e77c9-3428-45cb-a4be-7929a7e47a58 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:51.430579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904853e4-2cff-4126-b7d4-d7efcbd1998e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:51.432605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-904853e4-2cff-4126-b7d4-d7efcbd1998e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:51.455360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd68b11-3e58-4978-84b1-1503c398ef3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:51.456071 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd68b11-3e58-4978-84b1-1503c398ef3b admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:51.456346 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd68b11-3e58-4978-84b1-1503c398ef3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:51.456631 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd68b11-3e58-4978-84b1-1503c398ef3b admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:51.457333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd68b11-3e58-4978-84b1-1503c398ef3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:51.459309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd68b11-3e58-4978-84b1-1503c398ef3b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:51.490526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:51.501450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904853e4-2cff-4126-b7d4-d7efcbd1998e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:51.507702 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5522ca-ae2f-4382-9621-049b53aba980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:51.508383 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5522ca-ae2f-4382-9621-049b53aba980 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:51.508714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5522ca-ae2f-4382-9621-049b53aba980 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:51.509136 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5522ca-ae2f-4382-9621-049b53aba980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:51.511070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7f5522ca-ae2f-4382-9621-049b53aba980 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:51.555086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1059/2125] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:06:51 2022] POST /identity/v3/users => generated 291 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:51.559270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd68b11-3e58-4978-84b1-1503c398ef3b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:51.574066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd68b11-3e58-4978-84b1-1503c398ef3b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:51.581542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a5ccf-c7f3-4c8d-84cf-6bef793a1b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:51.582306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a5ccf-c7f3-4c8d-84cf-6bef793a1b40 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:51.582562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a5ccf-c7f3-4c8d-84cf-6bef793a1b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:51.582851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a5ccf-c7f3-4c8d-84cf-6bef793a1b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:51.594570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a5ccf-c7f3-4c8d-84cf-6bef793a1b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:51.594570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a5ccf-c7f3-4c8d-84cf-6bef793a1b40 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:51.594951 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5522ca-ae2f-4382-9621-049b53aba980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:51.594951 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5522ca-ae2f-4382-9621-049b53aba980 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:51.605033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1060/2126] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:06:51.607625 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1067/2127] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:06:51.610225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a5ccf-c7f3-4c8d-84cf-6bef793a1b40 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:51.624561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a5ccf-c7f3-4c8d-84cf-6bef793a1b40 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:51.630062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588793d9-5f66-41b6-96cf-0e23915fb6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:51.630908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588793d9-5f66-41b6-96cf-0e23915fb6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:51.631202 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588793d9-5f66-41b6-96cf-0e23915fb6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:51.631659 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588793d9-5f66-41b6-96cf-0e23915fb6d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:51.632731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588793d9-5f66-41b6-96cf-0e23915fb6d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:51.634620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588793d9-5f66-41b6-96cf-0e23915fb6d6 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:51.648193 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1061/2128] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:06:51.651506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5522ca-ae2f-4382-9621-049b53aba980 service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:51.663884 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5522ca-ae2f-4382-9621-049b53aba980 service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:51.668886 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cd157-0129-4795-a917-d705accfd84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:51.669811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cd157-0129-4795-a917-d705accfd84c admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:51.670098 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cd157-0129-4795-a917-d705accfd84c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:51.671064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cd157-0129-4795-a917-d705accfd84c admin admin] PATH_INFO: `/v3/projects/ffe03884791a4cb795e8e10341a04d95/users/b146639c09014f6ea15b742abc977492/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:51.691203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59cd157-0129-4795-a917-d705accfd84c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffe03884791a4cb795e8e10341a04d95, user_id=b146639c09014f6ea15b742abc977492, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:51.694109 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59cd157-0129-4795-a917-d705accfd84c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:51.746559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1068/2129] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:06:51 2022] PUT /identity/v3/projects/ffe03884791a4cb795e8e10341a04d95/users/b146639c09014f6ea15b742abc977492/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:06:51.759864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa6845-4bb5-4bdc-9fee-87d6749247f5 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:51.760164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa6845-4bb5-4bdc-9fee-87d6749247f5 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:51.760494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa6845-4bb5-4bdc-9fee-87d6749247f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:51.807609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c0fa6845-4bb5-4bdc-9fee-87d6749247f5 None None] MFA Rules not processed for user `b146639c09014f6ea15b742abc977492`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:51.808851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c0fa6845-4bb5-4bdc-9fee-87d6749247f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:51.898146 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1062/2130] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:06:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:06:51.909718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cd157-0129-4795-a917-d705accfd84c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:51.911664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f59cd157-0129-4795-a917-d705accfd84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:52.004563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cd157-0129-4795-a917-d705accfd84c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:52.010703 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d636e-4327-43b3-a3f9-645dd4ce70e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:52.011387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d636e-4327-43b3-a3f9-645dd4ce70e5 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:52.011663 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d636e-4327-43b3-a3f9-645dd4ce70e5 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:52.012025 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d636e-4327-43b3-a3f9-645dd4ce70e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:52.020539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d636e-4327-43b3-a3f9-645dd4ce70e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:52.022169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d636e-4327-43b3-a3f9-645dd4ce70e5 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:52.032675 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1069/2131] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:06:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:07:01.894669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:02.157237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=74,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:24.357971 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa6845-4bb5-4bdc-9fee-87d6749247f5 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:24.360676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c0fa6845-4bb5-4bdc-9fee-87d6749247f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:24.520633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa6845-4bb5-4bdc-9fee-87d6749247f5 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:24.542935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51737b9-b80d-4495-8d85-15bfc60dc2b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:24.544044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51737b9-b80d-4495-8d85-15bfc60dc2b0 service placement] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:24.544535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51737b9-b80d-4495-8d85-15bfc60dc2b0 service placement] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:24.545007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51737b9-b80d-4495-8d85-15bfc60dc2b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:24.550601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c51737b9-b80d-4495-8d85-15bfc60dc2b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:24.582723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51737b9-b80d-4495-8d85-15bfc60dc2b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:24.586360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51737b9-b80d-4495-8d85-15bfc60dc2b0 service placement] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:24.597951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1063/2132] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:07:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:07:25.781481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d636e-4327-43b3-a3f9-645dd4ce70e5 service nova] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:25.866191 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d636e-4327-43b3-a3f9-645dd4ce70e5 service nova] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:25.871440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:25.872360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:25.872643 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:25.872921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] PATH_INFO: `/v3/users/b146639c09014f6ea15b742abc977492` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:25.874605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=b146639c09014f6ea15b742abc977492)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:25.876519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:26.095319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b146639c09014f6ea15b742abc977492'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:26.096179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b146639c09014f6ea15b742abc977492'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:26.106491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b146639c09014f6ea15b742abc977492'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:26.126037 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b146639c09014f6ea15b742abc977492'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:26.236156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b146639c09014f6ea15b742abc977492'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:26.243697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1070/2133] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:07:25 2022] DELETE /identity/v3/users/b146639c09014f6ea15b742abc977492 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:26.248403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95860336-aa86-4ffb-90cd-8870240f4c6d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:26.248906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95860336-aa86-4ffb-90cd-8870240f4c6d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:26.249259 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95860336-aa86-4ffb-90cd-8870240f4c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:26.270813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-95860336-aa86-4ffb-90cd-8870240f4c6d None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:26.272067 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-95860336-aa86-4ffb-90cd-8870240f4c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:26.339851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1064/2134] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:07:26.350389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:26.360382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:26.413983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712232f-147a-483f-aae1-5badc0354bac admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:26.417680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547971f-b676-46dc-8e43-4cf678dff454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:26.418355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547971f-b676-46dc-8e43-4cf678dff454 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:26.418658 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547971f-b676-46dc-8e43-4cf678dff454 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:26.418939 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547971f-b676-46dc-8e43-4cf678dff454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:26.431027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547971f-b676-46dc-8e43-4cf678dff454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:26.432503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547971f-b676-46dc-8e43-4cf678dff454 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:26.445651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1071/2135] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:07:26.951789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95860336-aa86-4ffb-90cd-8870240f4c6d None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:26.953391 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-95860336-aa86-4ffb-90cd-8870240f4c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:26.966251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95860336-aa86-4ffb-90cd-8870240f4c6d None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:26.971618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:26.972437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:26.972723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:26.973007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735 admin admin] PATH_INFO: `/v3/projects/ffe03884791a4cb795e8e10341a04d95` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:26.974424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffe03884791a4cb795e8e10341a04d95)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:26.976480 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:27.111815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffe03884791a4cb795e8e10341a04d95', 'request_id': 'req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:27.206929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffe03884791a4cb795e8e10341a04d95', 'request_id': 'req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:27.213841 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffe03884791a4cb795e8e10341a04d95', 'request_id': 'req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:27.225848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735 admin admin] The token cache is being invalidate because project ffe03884791a4cb795e8e10341a04d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:07:27.226141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:27.226824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-517b5c1b-1c12-42ce-b4a0-4a68d8f43735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:27.228476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1065/2136] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:07:26 2022] DELETE /identity/v3/projects/ffe03884791a4cb795e8e10341a04d95 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:27.233426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d57802-9a30-461b-a6ed-75adb43a65c6 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:27.233909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d57802-9a30-461b-a6ed-75adb43a65c6 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:27.237237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d57802-9a30-461b-a6ed-75adb43a65c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:27.257615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-e7d57802-9a30-461b-a6ed-75adb43a65c6 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:27.259069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e7d57802-9a30-461b-a6ed-75adb43a65c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:27.290417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:27.350722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1072/2137] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:07:27.360494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895442ad-c009-47b1-bfae-5e206bbe07a9 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:27.361340 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895442ad-c009-47b1-bfae-5e206bbe07a9 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:27.361626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895442ad-c009-47b1-bfae-5e206bbe07a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:27.386488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-895442ad-c009-47b1-bfae-5e206bbe07a9 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:27.386488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-895442ad-c009-47b1-bfae-5e206bbe07a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:27.397137 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1066/2138] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:07:27.404435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d57802-9a30-461b-a6ed-75adb43a65c6 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:27.433573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d57802-9a30-461b-a6ed-75adb43a65c6 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:27.441399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e33065-a69e-487b-8a38-5b301a57464e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:27.442777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e33065-a69e-487b-8a38-5b301a57464e admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:27.444456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e33065-a69e-487b-8a38-5b301a57464e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:27.444456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e33065-a69e-487b-8a38-5b301a57464e admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:27.444456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e33065-a69e-487b-8a38-5b301a57464e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:27.445072 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e33065-a69e-487b-8a38-5b301a57464e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:27.446876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e33065-a69e-487b-8a38-5b301a57464e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:27.476857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1073/2139] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:27.482432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895442ad-c009-47b1-bfae-5e206bbe07a9 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:27.494736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895442ad-c009-47b1-bfae-5e206bbe07a9 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:27.500811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9529990-95c8-45e4-9e80-af264cc493b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:27.501856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9529990-95c8-45e4-9e80-af264cc493b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:27.502414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9529990-95c8-45e4-9e80-af264cc493b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:27.502849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9529990-95c8-45e4-9e80-af264cc493b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:27.503786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9529990-95c8-45e4-9e80-af264cc493b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:27.505872 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9529990-95c8-45e4-9e80-af264cc493b1 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:27.515211 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:27.621981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1067/2140] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:07:27 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:07:27.627215 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e33065-a69e-487b-8a38-5b301a57464e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:27.642671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e33065-a69e-487b-8a38-5b301a57464e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:27.695955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af42243-e158-404a-be40-cf795f5c200f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:27.696633 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af42243-e158-404a-be40-cf795f5c200f admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:27.696949 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af42243-e158-404a-be40-cf795f5c200f admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:27.697243 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af42243-e158-404a-be40-cf795f5c200f admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:27.697952 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af42243-e158-404a-be40-cf795f5c200f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:27.699966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af42243-e158-404a-be40-cf795f5c200f admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:27.798774 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1074/2141] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:07:27 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:27.804783 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9529990-95c8-45e4-9e80-af264cc493b1 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:27.819110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9529990-95c8-45e4-9e80-af264cc493b1 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:27.824138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a736b-3eb2-4069-9c75-bda31b86d496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:27.824816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a736b-3eb2-4069-9c75-bda31b86d496 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:27.825107 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a736b-3eb2-4069-9c75-bda31b86d496 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:27.825396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a736b-3eb2-4069-9c75-bda31b86d496 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:27.826133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a736b-3eb2-4069-9c75-bda31b86d496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:27.827808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a736b-3eb2-4069-9c75-bda31b86d496 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:27.839375 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1068/2142] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:07:27.844207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af42243-e158-404a-be40-cf795f5c200f admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:27.858528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af42243-e158-404a-be40-cf795f5c200f admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:27.864911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bf620-05cc-4bb6-87fc-71f8175b8900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:27.865844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bf620-05cc-4bb6-87fc-71f8175b8900 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:27.866135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bf620-05cc-4bb6-87fc-71f8175b8900 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:27.866474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bf620-05cc-4bb6-87fc-71f8175b8900 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:27.867216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bf620-05cc-4bb6-87fc-71f8175b8900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:27.868875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bf620-05cc-4bb6-87fc-71f8175b8900 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:27.882603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1075/2143] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:07:27.888693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a736b-3eb2-4069-9c75-bda31b86d496 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:27.900282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a736b-3eb2-4069-9c75-bda31b86d496 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:27.905185 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74a5dc-8e87-4e80-82b1-74ad18b4dd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:27.905991 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a5dc-8e87-4e80-82b1-74ad18b4dd31 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:27.906372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a5dc-8e87-4e80-82b1-74ad18b4dd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:27.906579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a5dc-8e87-4e80-82b1-74ad18b4dd31 admin admin] PATH_INFO: `/v3/projects/39ac7f88ad47491f80705e0b006b3d57/users/0223da004ef74317bfebd9b369858444/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:27.922286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74a5dc-8e87-4e80-82b1-74ad18b4dd31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ac7f88ad47491f80705e0b006b3d57, user_id=0223da004ef74317bfebd9b369858444, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:27.925003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74a5dc-8e87-4e80-82b1-74ad18b4dd31 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:27.997207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1069/2144] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:07:27 2022] PUT /identity/v3/projects/39ac7f88ad47491f80705e0b006b3d57/users/0223da004ef74317bfebd9b369858444/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:28.008587 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1e2f1-0f6f-4661-860b-68664320eb0a None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:28.008919 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1e2f1-0f6f-4661-860b-68664320eb0a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:28.009277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1e2f1-0f6f-4661-860b-68664320eb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:28.041862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-73f1e2f1-0f6f-4661-860b-68664320eb0a None None] MFA Rules not processed for user `0223da004ef74317bfebd9b369858444`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:28.043151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-73f1e2f1-0f6f-4661-860b-68664320eb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:28.123978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1076/2145] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:07:28.128985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74a5dc-8e87-4e80-82b1-74ad18b4dd31 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:28.205091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74a5dc-8e87-4e80-82b1-74ad18b4dd31 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:28.211679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da97d52-c3bc-40c0-b3cb-b89ce508b6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:28.211679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da97d52-c3bc-40c0-b3cb-b89ce508b6aa admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:28.211679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da97d52-c3bc-40c0-b3cb-b89ce508b6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:28.214843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da97d52-c3bc-40c0-b3cb-b89ce508b6aa admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:28.214843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da97d52-c3bc-40c0-b3cb-b89ce508b6aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:28.214843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da97d52-c3bc-40c0-b3cb-b89ce508b6aa admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:28.311699 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1070/2146] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:07:28 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:28.314712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1e2f1-0f6f-4661-860b-68664320eb0a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:28.327086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1e2f1-0f6f-4661-860b-68664320eb0a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:28.341629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:28.397337 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a067c-cf84-4064-9901-21ae8456724c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:28.399026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a067c-cf84-4064-9901-21ae8456724c admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:28.399801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a067c-cf84-4064-9901-21ae8456724c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:28.401588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a067c-cf84-4064-9901-21ae8456724c admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:28.402639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a067c-cf84-4064-9901-21ae8456724c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:28.410090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a067c-cf84-4064-9901-21ae8456724c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:28.523661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1077/2147] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:07:28 2022] POST /identity/v3/users => generated 295 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:28.528411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da97d52-c3bc-40c0-b3cb-b89ce508b6aa admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:28.545617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da97d52-c3bc-40c0-b3cb-b89ce508b6aa admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:28.551244 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0078ba-bfc5-4a1b-b01c-92300bcd2163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:28.551956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0078ba-bfc5-4a1b-b01c-92300bcd2163 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:28.552246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0078ba-bfc5-4a1b-b01c-92300bcd2163 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:28.552530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0078ba-bfc5-4a1b-b01c-92300bcd2163 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:28.553495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0078ba-bfc5-4a1b-b01c-92300bcd2163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:28.555182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0078ba-bfc5-4a1b-b01c-92300bcd2163 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:28.568423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1071/2148] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:07:28.572973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a067c-cf84-4064-9901-21ae8456724c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:28.586131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a067c-cf84-4064-9901-21ae8456724c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:28.591529 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc7da3-36e5-4db5-8419-f41e3fab437b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:28.592435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc7da3-36e5-4db5-8419-f41e3fab437b admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:28.592722 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc7da3-36e5-4db5-8419-f41e3fab437b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:28.593008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc7da3-36e5-4db5-8419-f41e3fab437b admin admin] PATH_INFO: `/v3/projects/fc68f50149004e8a8bd939a31c078360/users/456ba76c09ee47d1b025f9ef419c3162/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:28.610729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc7da3-36e5-4db5-8419-f41e3fab437b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc68f50149004e8a8bd939a31c078360, user_id=456ba76c09ee47d1b025f9ef419c3162, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:28.613538 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc7da3-36e5-4db5-8419-f41e3fab437b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:28.700108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1078/2149] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:07:28 2022] PUT /identity/v3/projects/fc68f50149004e8a8bd939a31c078360/users/456ba76c09ee47d1b025f9ef419c3162/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:07:28.704978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0078ba-bfc5-4a1b-b01c-92300bcd2163 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:28.715982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0078ba-bfc5-4a1b-b01c-92300bcd2163 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:28.786914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3abf9c-2a0f-4d66-bca5-b0a0e3715d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:28.787716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3abf9c-2a0f-4d66-bca5-b0a0e3715d2e admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:28.788189 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3abf9c-2a0f-4d66-bca5-b0a0e3715d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:28.788564 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3abf9c-2a0f-4d66-bca5-b0a0e3715d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:28.789531 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3abf9c-2a0f-4d66-bca5-b0a0e3715d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:28.791590 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3abf9c-2a0f-4d66-bca5-b0a0e3715d2e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:28.803423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1072/2150] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:07:28.813739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc7da3-36e5-4db5-8419-f41e3fab437b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:28.824597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc7da3-36e5-4db5-8419-f41e3fab437b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:28.830588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89db1f7-5bf0-4acf-a976-249fe28be389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:28.831491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89db1f7-5bf0-4acf-a976-249fe28be389 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:28.831780 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89db1f7-5bf0-4acf-a976-249fe28be389 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:28.832069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89db1f7-5bf0-4acf-a976-249fe28be389 admin admin] PATH_INFO: `/v3/projects/fc68f50149004e8a8bd939a31c078360/users/456ba76c09ee47d1b025f9ef419c3162/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:28.834192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89db1f7-5bf0-4acf-a976-249fe28be389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc68f50149004e8a8bd939a31c078360, user_id=456ba76c09ee47d1b025f9ef419c3162, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:28.836997 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89db1f7-5bf0-4acf-a976-249fe28be389 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:28.848613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1079/2151] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:07:28 2022] PUT /identity/v3/projects/fc68f50149004e8a8bd939a31c078360/users/456ba76c09ee47d1b025f9ef419c3162/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:28.861477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b98da-94c7-4b74-baaa-e71fd007bec1 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:28.867781 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b98da-94c7-4b74-baaa-e71fd007bec1 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:28.867781 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b98da-94c7-4b74-baaa-e71fd007bec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:28.892968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-090b98da-94c7-4b74-baaa-e71fd007bec1 None None] MFA Rules not processed for user `456ba76c09ee47d1b025f9ef419c3162`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:28.896932 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-090b98da-94c7-4b74-baaa-e71fd007bec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:28.904058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:28.975562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1073/2152] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:07:28.983277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89db1f7-5bf0-4acf-a976-249fe28be389 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:28.985294 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c89db1f7-5bf0-4acf-a976-249fe28be389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:29.053528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89db1f7-5bf0-4acf-a976-249fe28be389 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:29.057354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66f371-c6e2-492d-aa9f-aab0a9fed6d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:29.058042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66f371-c6e2-492d-aa9f-aab0a9fed6d3 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:29.058354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66f371-c6e2-492d-aa9f-aab0a9fed6d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:29.058634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66f371-c6e2-492d-aa9f-aab0a9fed6d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:29.126304 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66f371-c6e2-492d-aa9f-aab0a9fed6d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:29.127782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66f371-c6e2-492d-aa9f-aab0a9fed6d3 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:29.139214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1080/2153] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:07:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:07:29.521255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b98da-94c7-4b74-baaa-e71fd007bec1 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:29.523074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-090b98da-94c7-4b74-baaa-e71fd007bec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:29.644993 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b98da-94c7-4b74-baaa-e71fd007bec1 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:29.653401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:29.654306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:29.654634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:29.654899 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] PATH_INFO: `/v3/users/3ed7496968364068b430c3d878da3f45` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:29.657317 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ed7496968364068b430c3d878da3f45)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:29.660221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:30.002364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ed7496968364068b430c3d878da3f45'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:30.003400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ed7496968364068b430c3d878da3f45'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:30.004051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ed7496968364068b430c3d878da3f45'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:30.105905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ed7496968364068b430c3d878da3f45'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:30.153297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ed7496968364068b430c3d878da3f45'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:30.185008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1074/2154] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:07:29 2022] DELETE /identity/v3/users/3ed7496968364068b430c3d878da3f45 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:30.189900 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379923cd-34f6-4599-90e5-fff249f7a2a0 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:30.191008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379923cd-34f6-4599-90e5-fff249f7a2a0 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:30.191307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379923cd-34f6-4599-90e5-fff249f7a2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:30.219290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-379923cd-34f6-4599-90e5-fff249f7a2a0 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:30.223165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-379923cd-34f6-4599-90e5-fff249f7a2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:30.243136 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:30.295789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1081/2155] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:07:30.308069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:30.310033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:30.343253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:30.376181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd4f90-1b84-4406-b807-cb8b52b8274d admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:30.380257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb831ecf-993b-469f-9166-fdb95d23a645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:30.380936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb831ecf-993b-469f-9166-fdb95d23a645 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:30.381231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb831ecf-993b-469f-9166-fdb95d23a645 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:30.381918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb831ecf-993b-469f-9166-fdb95d23a645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:30.396202 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb831ecf-993b-469f-9166-fdb95d23a645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:30.397737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb831ecf-993b-469f-9166-fdb95d23a645 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:30.409468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1075/2156] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:07:30.935643 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379923cd-34f6-4599-90e5-fff249f7a2a0 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:30.938764 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-379923cd-34f6-4599-90e5-fff249f7a2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:30.971860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379923cd-34f6-4599-90e5-fff249f7a2a0 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:31.000648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:31.000648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:31.000648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:31.000648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] PATH_INFO: `/v3/projects/b17fe8a7474e4fbfa0bd16e5e879ef96` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:31.002907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b17fe8a7474e4fbfa0bd16e5e879ef96)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:31.003180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:31.130679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b17fe8a7474e4fbfa0bd16e5e879ef96', 'request_id': 'req-09b327f9-6f62-4cd1-bf0e-c24566074b0a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:31.148661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b17fe8a7474e4fbfa0bd16e5e879ef96', 'request_id': 'req-09b327f9-6f62-4cd1-bf0e-c24566074b0a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:31.161933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b17fe8a7474e4fbfa0bd16e5e879ef96', 'request_id': 'req-09b327f9-6f62-4cd1-bf0e-c24566074b0a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:31.243147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] The token cache is being invalidate because project b17fe8a7474e4fbfa0bd16e5e879ef96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:07:31.243749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:31.244624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:31.254545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1082/2157] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:07:30 2022] DELETE /identity/v3/projects/b17fe8a7474e4fbfa0bd16e5e879ef96 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:39.858078 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb831ecf-993b-469f-9166-fdb95d23a645 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:39.860500 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-cb831ecf-993b-469f-9166-fdb95d23a645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:39.933529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb831ecf-993b-469f-9166-fdb95d23a645 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:39.997654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d86f64-3457-4af1-9e0d-dc8853e379af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:39.998286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d86f64-3457-4af1-9e0d-dc8853e379af service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:39.998712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d86f64-3457-4af1-9e0d-dc8853e379af service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:39.998839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d86f64-3457-4af1-9e0d-dc8853e379af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:40.001672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-46d86f64-3457-4af1-9e0d-dc8853e379af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:40.073671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d86f64-3457-4af1-9e0d-dc8853e379af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:40.075265 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d86f64-3457-4af1-9e0d-dc8853e379af service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:40.086940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1076/2158] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:07:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:07:40.809057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:40.810682 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:40.928001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b327f9-6f62-4cd1-bf0e-c24566074b0a admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:40.938954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:40.938954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:40.938954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:40.938954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] PATH_INFO: `/v3/users/6f07dd65478149209d8b8306e04ba980` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:40.949151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f07dd65478149209d8b8306e04ba980)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:40.951905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:41.182840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f07dd65478149209d8b8306e04ba980'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:41.183607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f07dd65478149209d8b8306e04ba980'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:41.186018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:41.191840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f07dd65478149209d8b8306e04ba980'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:41.205436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f07dd65478149209d8b8306e04ba980'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:41.286285 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f07dd65478149209d8b8306e04ba980'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:41.293357 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1083/2159] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:07:40 2022] DELETE /identity/v3/users/6f07dd65478149209d8b8306e04ba980 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:41.562982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d86f64-3457-4af1-9e0d-dc8853e379af service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:41.575999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d86f64-3457-4af1-9e0d-dc8853e379af service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:41.620672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:41.660969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:41.662561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:41.663043 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:41.663529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] PATH_INFO: `/v3/projects/02e0f85bb74149c0a452d19d33fe8194` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:41.665696 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02e0f85bb74149c0a452d19d33fe8194)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:41.669595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:41.802081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02e0f85bb74149c0a452d19d33fe8194', 'request_id': 'req-1366e745-21dd-4ec1-bbe0-fead3be4ac69'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:41.876835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02e0f85bb74149c0a452d19d33fe8194', 'request_id': 'req-1366e745-21dd-4ec1-bbe0-fead3be4ac69'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:41.884679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02e0f85bb74149c0a452d19d33fe8194', 'request_id': 'req-1366e745-21dd-4ec1-bbe0-fead3be4ac69'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:41.899214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] The token cache is being invalidate because project 02e0f85bb74149c0a452d19d33fe8194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:07:41.899501 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:41.900924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:41.902638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1077/2160] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:07:41 2022] DELETE /identity/v3/projects/02e0f85bb74149c0a452d19d33fe8194 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:41.906608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:41.908068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:41.922992 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c18fe-fa8b-4838-938f-8f61a27cb9fe admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:41.997153 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f7931-8e06-4e9c-b2e2-55b1836c680b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:41.998243 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f7931-8e06-4e9c-b2e2-55b1836c680b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:41.998591 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f7931-8e06-4e9c-b2e2-55b1836c680b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:41.998877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f7931-8e06-4e9c-b2e2-55b1836c680b admin admin] PATH_INFO: `/v3/users/718447055e6f412397b8c7c7a2f51122` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:42.000401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f7931-8e06-4e9c-b2e2-55b1836c680b admin admin] RBAC: Authorizing `identity:delete_user(user_id=718447055e6f412397b8c7c7a2f51122)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:42.002504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f7931-8e06-4e9c-b2e2-55b1836c680b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:42.275690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-080f7931-8e06-4e9c-b2e2-55b1836c680b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '718447055e6f412397b8c7c7a2f51122'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:42.276402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-080f7931-8e06-4e9c-b2e2-55b1836c680b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '718447055e6f412397b8c7c7a2f51122'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:42.285844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-080f7931-8e06-4e9c-b2e2-55b1836c680b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '718447055e6f412397b8c7c7a2f51122'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:42.300718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-080f7931-8e06-4e9c-b2e2-55b1836c680b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '718447055e6f412397b8c7c7a2f51122'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:42.361057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-080f7931-8e06-4e9c-b2e2-55b1836c680b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '718447055e6f412397b8c7c7a2f51122'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:42.367973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1084/2161] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:07:41 2022] DELETE /identity/v3/users/718447055e6f412397b8c7c7a2f51122 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:07:42.653576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:42.655242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:42.668673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366e745-21dd-4ec1-bbe0-fead3be4ac69 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:42.744903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b95dc-5c77-4d39-b360-3d162d22bf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:42.745867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b95dc-5c77-4d39-b360-3d162d22bf61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:42.746163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b95dc-5c77-4d39-b360-3d162d22bf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:42.746487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b95dc-5c77-4d39-b360-3d162d22bf61 admin admin] PATH_INFO: `/v3/projects/510a2d4c29044244b973a54e712b31f9` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:42.747819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b95dc-5c77-4d39-b360-3d162d22bf61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=510a2d4c29044244b973a54e712b31f9)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:42.750298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b95dc-5c77-4d39-b360-3d162d22bf61 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:42.850873 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-232b95dc-5c77-4d39-b360-3d162d22bf61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '510a2d4c29044244b973a54e712b31f9', 'request_id': 'req-232b95dc-5c77-4d39-b360-3d162d22bf61'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:42.914960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-232b95dc-5c77-4d39-b360-3d162d22bf61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '510a2d4c29044244b973a54e712b31f9', 'request_id': 'req-232b95dc-5c77-4d39-b360-3d162d22bf61'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:42.922010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-232b95dc-5c77-4d39-b360-3d162d22bf61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '510a2d4c29044244b973a54e712b31f9', 'request_id': 'req-232b95dc-5c77-4d39-b360-3d162d22bf61'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:42.925665 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:42.933960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-232b95dc-5c77-4d39-b360-3d162d22bf61 admin admin] The token cache is being invalidate because project 510a2d4c29044244b973a54e712b31f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:07:42.934252 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-232b95dc-5c77-4d39-b360-3d162d22bf61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:42.935895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-232b95dc-5c77-4d39-b360-3d162d22bf61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:42.937585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1078/2162] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:07:42 2022] DELETE /identity/v3/projects/510a2d4c29044244b973a54e712b31f9 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:07:42.950007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00ac2-429f-465f-87fe-9c9ed742705e None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:42.950007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00ac2-429f-465f-87fe-9c9ed742705e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:42.950007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00ac2-429f-465f-87fe-9c9ed742705e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:42.977829 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d7c00ac2-429f-465f-87fe-9c9ed742705e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:42.979963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d7c00ac2-429f-465f-87fe-9c9ed742705e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:43.040856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:43.060616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1085/2163] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:07:43.070688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5703c-498f-409b-b494-758aa3762aa1 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.071036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5703c-498f-409b-b494-758aa3762aa1 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.071334 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5703c-498f-409b-b494-758aa3762aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.092389 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-57b5703c-498f-409b-b494-758aa3762aa1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:43.094664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-57b5703c-498f-409b-b494-758aa3762aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:43.102951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1079/2164] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:07:43.107488 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c00ac2-429f-465f-87fe-9c9ed742705e None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:43.118062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c00ac2-429f-465f-87fe-9c9ed742705e None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:43.122891 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41baddf4-84d0-4058-998a-3e5660ef87e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:43.123551 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41baddf4-84d0-4058-998a-3e5660ef87e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.123962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41baddf4-84d0-4058-998a-3e5660ef87e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.124107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41baddf4-84d0-4058-998a-3e5660ef87e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.125284 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41baddf4-84d0-4058-998a-3e5660ef87e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:43.125560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41baddf4-84d0-4058-998a-3e5660ef87e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:43.127220 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41baddf4-84d0-4058-998a-3e5660ef87e2 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:43.139263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1086/2165] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:43.145100 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5703c-498f-409b-b494-758aa3762aa1 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:43.155245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5703c-498f-409b-b494-758aa3762aa1 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:43.161632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c0b0a-2406-4266-8a3e-a30db1497ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:43.162450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c0b0a-2406-4266-8a3e-a30db1497ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.162865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c0b0a-2406-4266-8a3e-a30db1497ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.163439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c0b0a-2406-4266-8a3e-a30db1497ac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.164127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c0b0a-2406-4266-8a3e-a30db1497ac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:43.166222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c0b0a-2406-4266-8a3e-a30db1497ac1 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:43.276429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1080/2166] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:07:43 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:43.277741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41baddf4-84d0-4058-998a-3e5660ef87e2 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:43.303891 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41baddf4-84d0-4058-998a-3e5660ef87e2 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:43.365373 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473cf9e-d1df-415f-b644-224b3331957d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:43.366059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473cf9e-d1df-415f-b644-224b3331957d admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.366516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473cf9e-d1df-415f-b644-224b3331957d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.367048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473cf9e-d1df-415f-b644-224b3331957d admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.367782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473cf9e-d1df-415f-b644-224b3331957d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:43.369749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473cf9e-d1df-415f-b644-224b3331957d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:43.442235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1087/2167] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:07:43 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:07:43.446949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c0b0a-2406-4266-8a3e-a30db1497ac1 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:43.461011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c0b0a-2406-4266-8a3e-a30db1497ac1 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:43.477435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b072c-ef5b-4de5-9868-1c7a85bfca11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:43.478776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b072c-ef5b-4de5-9868-1c7a85bfca11 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.478776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b072c-ef5b-4de5-9868-1c7a85bfca11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.478776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b072c-ef5b-4de5-9868-1c7a85bfca11 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.479979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b072c-ef5b-4de5-9868-1c7a85bfca11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:43.481064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b072c-ef5b-4de5-9868-1c7a85bfca11 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:43.498076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1081/2168] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:07:43.500089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473cf9e-d1df-415f-b644-224b3331957d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:43.512606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473cf9e-d1df-415f-b644-224b3331957d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:43.518108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96116480-dd75-474a-8e9c-279c91d95f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:43.518862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96116480-dd75-474a-8e9c-279c91d95f83 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.519199 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96116480-dd75-474a-8e9c-279c91d95f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.519429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96116480-dd75-474a-8e9c-279c91d95f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.520182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96116480-dd75-474a-8e9c-279c91d95f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:43.521761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96116480-dd75-474a-8e9c-279c91d95f83 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:43.532551 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1088/2169] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:07:43.536811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b072c-ef5b-4de5-9868-1c7a85bfca11 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:43.547255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b072c-ef5b-4de5-9868-1c7a85bfca11 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:43.552000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3691c6-6637-4322-9033-e045f7bdcdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:43.552886 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3691c6-6637-4322-9033-e045f7bdcdc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.553206 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3691c6-6637-4322-9033-e045f7bdcdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.553500 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3691c6-6637-4322-9033-e045f7bdcdc0 admin admin] PATH_INFO: `/v3/projects/65833cba7b814fe5ab7e67fad9865a43/users/ed38a29cda574ea486e68e75e5843dba/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.569532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3691c6-6637-4322-9033-e045f7bdcdc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65833cba7b814fe5ab7e67fad9865a43, user_id=ed38a29cda574ea486e68e75e5843dba, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:43.572384 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3691c6-6637-4322-9033-e045f7bdcdc0 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:43.653413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1082/2170] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:07:43 2022] PUT /identity/v3/projects/65833cba7b814fe5ab7e67fad9865a43/users/ed38a29cda574ea486e68e75e5843dba/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:07:43.664012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1f5d9-9dd8-47f0-8be8-672faa4895f3 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.664300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1f5d9-9dd8-47f0-8be8-672faa4895f3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.669413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1f5d9-9dd8-47f0-8be8-672faa4895f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.698667 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-e2b1f5d9-9dd8-47f0-8be8-672faa4895f3 None None] MFA Rules not processed for user `ed38a29cda574ea486e68e75e5843dba`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:43.699785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e2b1f5d9-9dd8-47f0-8be8-672faa4895f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:43.767178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1089/2171] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:07:43.779048 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3691c6-6637-4322-9033-e045f7bdcdc0 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:43.780957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-cd3691c6-6637-4322-9033-e045f7bdcdc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:43.839035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3691c6-6637-4322-9033-e045f7bdcdc0 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:43.843257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cf29f-9279-444f-9972-a43d968f6c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:43.843992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cf29f-9279-444f-9972-a43d968f6c05 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.844282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cf29f-9279-444f-9972-a43d968f6c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.844636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cf29f-9279-444f-9972-a43d968f6c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.853482 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cf29f-9279-444f-9972-a43d968f6c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:43.855046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cf29f-9279-444f-9972-a43d968f6c05 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:43.864611 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1083/2172] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:07:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:07:52.400712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1f5d9-9dd8-47f0-8be8-672faa4895f3 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:52.473315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:52.494201 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1f5d9-9dd8-47f0-8be8-672faa4895f3 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:52.498873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d963286-7a56-4dd9-9f5d-0be8774555dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:52.500389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d963286-7a56-4dd9-9f5d-0be8774555dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:52.500675 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d963286-7a56-4dd9-9f5d-0be8774555dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:52.500951 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d963286-7a56-4dd9-9f5d-0be8774555dd admin admin] PATH_INFO: `/v3/users/ed38a29cda574ea486e68e75e5843dba` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:52.502210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d963286-7a56-4dd9-9f5d-0be8774555dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed38a29cda574ea486e68e75e5843dba)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:52.515105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d963286-7a56-4dd9-9f5d-0be8774555dd admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:53.034514 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-3d963286-7a56-4dd9-9f5d-0be8774555dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed38a29cda574ea486e68e75e5843dba'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:53.035229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-3d963286-7a56-4dd9-9f5d-0be8774555dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed38a29cda574ea486e68e75e5843dba'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:53.049389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-3d963286-7a56-4dd9-9f5d-0be8774555dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed38a29cda574ea486e68e75e5843dba'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:53.075416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-3d963286-7a56-4dd9-9f5d-0be8774555dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed38a29cda574ea486e68e75e5843dba'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:53.143741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-3d963286-7a56-4dd9-9f5d-0be8774555dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed38a29cda574ea486e68e75e5843dba'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:53.159041 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1090/2173] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:07:52 2022] DELETE /identity/v3/users/ed38a29cda574ea486e68e75e5843dba => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:53.164632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8d631-d405-4cbf-943e-ab9389e88b0d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:53.165224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8d631-d405-4cbf-943e-ab9389e88b0d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:53.165409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8d631-d405-4cbf-943e-ab9389e88b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:53.189364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-8de8d631-d405-4cbf-943e-ab9389e88b0d None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:53.190726 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8de8d631-d405-4cbf-943e-ab9389e88b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:53.245610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af611b02-704a-4ce8-af00-a9bbbda0a615 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:53.246297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af611b02-704a-4ce8-af00-a9bbbda0a615 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:53.247001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af611b02-704a-4ce8-af00-a9bbbda0a615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:53.273111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-af611b02-704a-4ce8-af00-a9bbbda0a615 None None] MFA Rules not processed for user `7e3f168283ba4f159948830f396ae7cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:53.274620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-af611b02-704a-4ce8-af00-a9bbbda0a615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:53.313824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1084/2174] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:07:53.329427 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8d631-d405-4cbf-943e-ab9389e88b0d None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:53.332422 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8de8d631-d405-4cbf-943e-ab9389e88b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:53.382745 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:53.385413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1091/2175] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:07:53.412946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8d631-d405-4cbf-943e-ab9389e88b0d None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:53.416901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6e514-afad-48f9-9122-44112a4a1c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:53.417568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6e514-afad-48f9-9122-44112a4a1c3b service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:53.417856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6e514-afad-48f9-9122-44112a4a1c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:53.418408 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6e514-afad-48f9-9122-44112a4a1c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:53.434473 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6e514-afad-48f9-9122-44112a4a1c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:53.437206 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6e514-afad-48f9-9122-44112a4a1c3b service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:53.451229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1085/2176] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:07:54.302640 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af611b02-704a-4ce8-af00-a9bbbda0a615 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:54.304681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-af611b02-704a-4ce8-af00-a9bbbda0a615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:54.323167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af611b02-704a-4ce8-af00-a9bbbda0a615 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:54.329344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:54.330694 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:54.330694 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:54.331054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf admin admin] PATH_INFO: `/v3/projects/65833cba7b814fe5ab7e67fad9865a43` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:54.332278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=65833cba7b814fe5ab7e67fad9865a43)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:54.334914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:54.455964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65833cba7b814fe5ab7e67fad9865a43', 'request_id': 'req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:54.463773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65833cba7b814fe5ab7e67fad9865a43', 'request_id': 'req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:54.470176 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:54.471653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65833cba7b814fe5ab7e67fad9865a43', 'request_id': 'req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:54.579681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf admin admin] The token cache is being invalidate because project 65833cba7b814fe5ab7e67fad9865a43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:07:54.579981 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:54.580769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2066e3b0-79bd-485e-a8a5-0db7d1672fbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:07:54.587963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1092/2177] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:07:54 2022] DELETE /identity/v3/projects/65833cba7b814fe5ab7e67fad9865a43 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:54.593672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc91a3-97bb-4a9d-a9dc-be43faa4aaaa None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:54.594658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc91a3-97bb-4a9d-a9dc-be43faa4aaaa None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:54.595173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc91a3-97bb-4a9d-a9dc-be43faa4aaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:54.619268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b6dc91a3-97bb-4a9d-a9dc-be43faa4aaaa None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:54.620524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b6dc91a3-97bb-4a9d-a9dc-be43faa4aaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:54.697910 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1086/2178] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:07:54.706676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21153163-3066-4e6a-8b1f-19e07f80d0cb None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:54.707052 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21153163-3066-4e6a-8b1f-19e07f80d0cb None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:54.707592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21153163-3066-4e6a-8b1f-19e07f80d0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:54.730457 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-21153163-3066-4e6a-8b1f-19e07f80d0cb None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:54.731692 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-21153163-3066-4e6a-8b1f-19e07f80d0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:54.741291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1093/2179] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:07:54.746689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc91a3-97bb-4a9d-a9dc-be43faa4aaaa None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:54.764140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc91a3-97bb-4a9d-a9dc-be43faa4aaaa None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:54.769842 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688f66a-eb4b-488c-a3e8-7cba0cf91728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:54.770540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688f66a-eb4b-488c-a3e8-7cba0cf91728 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:54.770831 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688f66a-eb4b-488c-a3e8-7cba0cf91728 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:54.771114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688f66a-eb4b-488c-a3e8-7cba0cf91728 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:54.771852 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688f66a-eb4b-488c-a3e8-7cba0cf91728 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:54.772164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688f66a-eb4b-488c-a3e8-7cba0cf91728 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:54.773791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688f66a-eb4b-488c-a3e8-7cba0cf91728 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:54.787139 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1087/2180] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:54.791592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21153163-3066-4e6a-8b1f-19e07f80d0cb None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:54.802935 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21153163-3066-4e6a-8b1f-19e07f80d0cb None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:54.809117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2098ab-697d-4390-86a5-7349f6f506ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:54.809932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2098ab-697d-4390-86a5-7349f6f506ac admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:54.810312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2098ab-697d-4390-86a5-7349f6f506ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:54.810720 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2098ab-697d-4390-86a5-7349f6f506ac admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:54.811691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2098ab-697d-4390-86a5-7349f6f506ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:54.814018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2098ab-697d-4390-86a5-7349f6f506ac admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:54.980259 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1094/2181] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:07:54 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:54.984757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688f66a-eb4b-488c-a3e8-7cba0cf91728 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:54.996902 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688f66a-eb4b-488c-a3e8-7cba0cf91728 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:55.063837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75cae3-038c-4a0b-9c08-b06de6c8ec3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:55.064532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75cae3-038c-4a0b-9c08-b06de6c8ec3b admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:55.064815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75cae3-038c-4a0b-9c08-b06de6c8ec3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:55.065096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75cae3-038c-4a0b-9c08-b06de6c8ec3b admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:55.065835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75cae3-038c-4a0b-9c08-b06de6c8ec3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:55.067811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75cae3-038c-4a0b-9c08-b06de6c8ec3b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:55.163746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1088/2182] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:07:54 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:55.166877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2098ab-697d-4390-86a5-7349f6f506ac admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:55.178525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2098ab-697d-4390-86a5-7349f6f506ac admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:55.183442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6406c4-7c46-40ab-9f6f-3e634f62acdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:55.184153 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6406c4-7c46-40ab-9f6f-3e634f62acdf admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:55.184442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6406c4-7c46-40ab-9f6f-3e634f62acdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:55.184727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6406c4-7c46-40ab-9f6f-3e634f62acdf admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:55.185469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6406c4-7c46-40ab-9f6f-3e634f62acdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:55.187331 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6406c4-7c46-40ab-9f6f-3e634f62acdf admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:55.200053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1095/2183] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:07:55.206862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75cae3-038c-4a0b-9c08-b06de6c8ec3b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:55.218382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75cae3-038c-4a0b-9c08-b06de6c8ec3b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:55.223858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f49422-3144-4b20-aee9-923911257069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:55.224711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f49422-3144-4b20-aee9-923911257069 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:55.225000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f49422-3144-4b20-aee9-923911257069 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:55.225281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f49422-3144-4b20-aee9-923911257069 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:55.226032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f49422-3144-4b20-aee9-923911257069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:55.227955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f49422-3144-4b20-aee9-923911257069 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:55.242421 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1089/2184] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:07:55.246729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6406c4-7c46-40ab-9f6f-3e634f62acdf admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:55.258519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6406c4-7c46-40ab-9f6f-3e634f62acdf admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:55.263475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856c914-730f-4947-9c26-163f3a792892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:55.264381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856c914-730f-4947-9c26-163f3a792892 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:55.265231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856c914-730f-4947-9c26-163f3a792892 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:55.265512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856c914-730f-4947-9c26-163f3a792892 admin admin] PATH_INFO: `/v3/projects/4e489f01d3eb4d57847fd4153d42f620/users/6612b84e3ddb418d8f8d4c4ec7c10a8f/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:55.285702 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856c914-730f-4947-9c26-163f3a792892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e489f01d3eb4d57847fd4153d42f620, user_id=6612b84e3ddb418d8f8d4c4ec7c10a8f, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:55.289595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856c914-730f-4947-9c26-163f3a792892 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:55.425392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1096/2185] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:07:55 2022] PUT /identity/v3/projects/4e489f01d3eb4d57847fd4153d42f620/users/6612b84e3ddb418d8f8d4c4ec7c10a8f/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:55.441007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b4f22-d6d4-4a80-9bd6-300d4e004ec4 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:55.441300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b4f22-d6d4-4a80-9bd6-300d4e004ec4 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:55.441591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b4f22-d6d4-4a80-9bd6-300d4e004ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:55.473143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-012b4f22-d6d4-4a80-9bd6-300d4e004ec4 None None] MFA Rules not processed for user `6612b84e3ddb418d8f8d4c4ec7c10a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:55.474499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-012b4f22-d6d4-4a80-9bd6-300d4e004ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:55.507469 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:55.552362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1090/2186] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:07:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:07:55.566172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856c914-730f-4947-9c26-163f3a792892 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:55.568130 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f856c914-730f-4947-9c26-163f3a792892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:55.645627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856c914-730f-4947-9c26-163f3a792892 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:55.655973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c0dd2-f747-4283-bd58-c5b54d9d9bec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:55.655973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c0dd2-f747-4283-bd58-c5b54d9d9bec service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:55.655973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c0dd2-f747-4283-bd58-c5b54d9d9bec service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:55.655973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c0dd2-f747-4283-bd58-c5b54d9d9bec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:55.667211 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c0dd2-f747-4283-bd58-c5b54d9d9bec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:55.668666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c0dd2-f747-4283-bd58-c5b54d9d9bec service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:55.684302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1097/2187] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:07:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:08:05.675002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=13 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:05.675956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=56,keystone:INSERT=3,keystone:DELETE=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:17.983468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c7d55-4878-47a5-8d66-e1ae29422057 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:17.992429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c7d55-4878-47a5-8d66-e1ae29422057 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:17.992827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c7d55-4878-47a5-8d66-e1ae29422057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:18.030662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-068c7d55-4878-47a5-8d66-e1ae29422057 None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:18.039641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-068c7d55-4878-47a5-8d66-e1ae29422057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:18.119319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1091/2188] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:08:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:08:18.179307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c0dd2-f747-4283-bd58-c5b54d9d9bec service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:18.180951 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-cd4c0dd2-f747-4283-bd58-c5b54d9d9bec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:18.262807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c0dd2-f747-4283-bd58-c5b54d9d9bec service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:18.269699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c892a5-59cf-45ef-ae6e-8d835033cb55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:18.270747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c892a5-59cf-45ef-ae6e-8d835033cb55 service swift] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:18.271143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c892a5-59cf-45ef-ae6e-8d835033cb55 service swift] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:18.271554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c892a5-59cf-45ef-ae6e-8d835033cb55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:18.285127 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c892a5-59cf-45ef-ae6e-8d835033cb55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:18.286797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c892a5-59cf-45ef-ae6e-8d835033cb55 service swift] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:18.296686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1098/2189] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:08:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:08:28.105616 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=25 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:28.292534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=32 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:43.786409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20609332-3fb4-4b35-8d2c-9e86ea5f9153 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:43.787000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20609332-3fb4-4b35-8d2c-9e86ea5f9153 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:43.787000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20609332-3fb4-4b35-8d2c-9e86ea5f9153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:43.808617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-20609332-3fb4-4b35-8d2c-9e86ea5f9153 None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:43.810293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-20609332-3fb4-4b35-8d2c-9e86ea5f9153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:43.818927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1092/2190] 10.222.0.17 () {58 vars in 967 bytes} [Mon Jul 25 19:08:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:08:43.831926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c892a5-59cf-45ef-ae6e-8d835033cb55 service swift] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:43.842765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c892a5-59cf-45ef-ae6e-8d835033cb55 service swift] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:43.848042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a5433-7d26-42ac-8191-4e4c2afc90de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:43.853398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a5433-7d26-42ac-8191-4e4c2afc90de service swift] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:43.853398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a5433-7d26-42ac-8191-4e4c2afc90de service swift] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:43.853398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a5433-7d26-42ac-8191-4e4c2afc90de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:43.857867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a5433-7d26-42ac-8191-4e4c2afc90de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:43.859485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a5433-7d26-42ac-8191-4e4c2afc90de service swift] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:43.868695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1099/2191] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:08:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:08:50.092949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20609332-3fb4-4b35-8d2c-9e86ea5f9153 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:50.094752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-20609332-3fb4-4b35-8d2c-9e86ea5f9153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:50.185036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20609332-3fb4-4b35-8d2c-9e86ea5f9153 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:50.190561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b24643-91ba-4e0a-bb79-4be7e057bbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:50.191474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b24643-91ba-4e0a-bb79-4be7e057bbe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:50.191880 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b24643-91ba-4e0a-bb79-4be7e057bbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:50.192095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b24643-91ba-4e0a-bb79-4be7e057bbe7 admin admin] PATH_INFO: `/v3/users/8d04f1378edc435a801a6c2719ec31f5` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:50.193515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b24643-91ba-4e0a-bb79-4be7e057bbe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d04f1378edc435a801a6c2719ec31f5)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:50.195566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b24643-91ba-4e0a-bb79-4be7e057bbe7 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:50.388509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a5433-7d26-42ac-8191-4e4c2afc90de service swift] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:50.389994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-687a5433-7d26-42ac-8191-4e4c2afc90de service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:50.405343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a5433-7d26-42ac-8191-4e4c2afc90de service swift] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:50.411112 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:50.411993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:50.414431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:50.414431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] PATH_INFO: `/v3/users/0223da004ef74317bfebd9b369858444` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:50.414431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0223da004ef74317bfebd9b369858444)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:50.415787 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:50.455419 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-08b24643-91ba-4e0a-bb79-4be7e057bbe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d04f1378edc435a801a6c2719ec31f5'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:50.456147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-08b24643-91ba-4e0a-bb79-4be7e057bbe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d04f1378edc435a801a6c2719ec31f5'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:50.456852 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-08b24643-91ba-4e0a-bb79-4be7e057bbe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d04f1378edc435a801a6c2719ec31f5'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:50.554304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-08b24643-91ba-4e0a-bb79-4be7e057bbe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d04f1378edc435a801a6c2719ec31f5'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:50.569486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-08b24643-91ba-4e0a-bb79-4be7e057bbe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d04f1378edc435a801a6c2719ec31f5'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:50.582939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1093/2192] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:08:50 2022] DELETE /identity/v3/users/8d04f1378edc435a801a6c2719ec31f5 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:50.630865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0223da004ef74317bfebd9b369858444'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:50.631693 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0223da004ef74317bfebd9b369858444'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:50.639824 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0223da004ef74317bfebd9b369858444'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:50.654350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0223da004ef74317bfebd9b369858444'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:50.748600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0223da004ef74317bfebd9b369858444'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:50.756585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1100/2193] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:08:50 2022] DELETE /identity/v3/users/0223da004ef74317bfebd9b369858444 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:08:50.761157 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f498b5-a4c0-46a9-b577-57f525ce75b5 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:50.761459 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f498b5-a4c0-46a9-b577-57f525ce75b5 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:50.761740 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f498b5-a4c0-46a9-b577-57f525ce75b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:50.782063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-99f498b5-a4c0-46a9-b577-57f525ce75b5 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:50.783322 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-99f498b5-a4c0-46a9-b577-57f525ce75b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:50.849772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1094/2194] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:08:50.853208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:50.855014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:50.865723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f498b5-a4c0-46a9-b577-57f525ce75b5 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:50.868187 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-99f498b5-a4c0-46a9-b577-57f525ce75b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:50.871632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf26ac6-fb5a-4b35-987c-662ada469c5a admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:50.877683 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072da78-4343-478c-a617-e646a021b7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:50.879191 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072da78-4343-478c-a617-e646a021b7f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:50.879191 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072da78-4343-478c-a617-e646a021b7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:50.879191 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072da78-4343-478c-a617-e646a021b7f9 admin admin] PATH_INFO: `/v3/projects/9062c0ee33bb464c88d5f5e7471c6961` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:50.880642 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072da78-4343-478c-a617-e646a021b7f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9062c0ee33bb464c88d5f5e7471c6961)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:50.882639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072da78-4343-478c-a617-e646a021b7f9 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:50.890071 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:50.942496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f498b5-a4c0-46a9-b577-57f525ce75b5 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:50.946215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716f31c-925a-4211-8071-600915c11527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:50.946934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716f31c-925a-4211-8071-600915c11527 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:50.947289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716f31c-925a-4211-8071-600915c11527 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:50.947572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716f31c-925a-4211-8071-600915c11527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:50.958054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716f31c-925a-4211-8071-600915c11527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:50.959570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716f31c-925a-4211-8071-600915c11527 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:50.970248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1095/2195] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:08:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:08:51.009495 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2072da78-4343-478c-a617-e646a021b7f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9062c0ee33bb464c88d5f5e7471c6961', 'request_id': 'req-2072da78-4343-478c-a617-e646a021b7f9'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:51.017940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2072da78-4343-478c-a617-e646a021b7f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9062c0ee33bb464c88d5f5e7471c6961', 'request_id': 'req-2072da78-4343-478c-a617-e646a021b7f9'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:51.024586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2072da78-4343-478c-a617-e646a021b7f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9062c0ee33bb464c88d5f5e7471c6961', 'request_id': 'req-2072da78-4343-478c-a617-e646a021b7f9'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:51.147634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2072da78-4343-478c-a617-e646a021b7f9 admin admin] The token cache is being invalidate because project 9062c0ee33bb464c88d5f5e7471c6961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:08:51.147924 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2072da78-4343-478c-a617-e646a021b7f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:51.148574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2072da78-4343-478c-a617-e646a021b7f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:51.150156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1101/2196] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:08:50 2022] DELETE /identity/v3/projects/9062c0ee33bb464c88d5f5e7471c6961 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:51.155889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae99e7-70fd-4bb1-83aa-5602728e3b17 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.156439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae99e7-70fd-4bb1-83aa-5602728e3b17 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.156439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae99e7-70fd-4bb1-83aa-5602728e3b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.178063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-16ae99e7-70fd-4bb1-83aa-5602728e3b17 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:51.180462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-16ae99e7-70fd-4bb1-83aa-5602728e3b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:51.260274 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1096/2197] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:08:51.269778 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67efea6-1134-451a-97b9-7d3a90f5a446 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.270069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67efea6-1134-451a-97b9-7d3a90f5a446 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.270381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67efea6-1134-451a-97b9-7d3a90f5a446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.294662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-a67efea6-1134-451a-97b9-7d3a90f5a446 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:51.296092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a67efea6-1134-451a-97b9-7d3a90f5a446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:51.306611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1102/2198] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:08:51.312721 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae99e7-70fd-4bb1-83aa-5602728e3b17 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.326811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae99e7-70fd-4bb1-83aa-5602728e3b17 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.335062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084ca7f-8e04-48d1-9565-c1cbd064296f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.335751 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084ca7f-8e04-48d1-9565-c1cbd064296f admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.336042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084ca7f-8e04-48d1-9565-c1cbd064296f admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.336326 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084ca7f-8e04-48d1-9565-c1cbd064296f admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.337070 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084ca7f-8e04-48d1-9565-c1cbd064296f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:51.337389 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084ca7f-8e04-48d1-9565-c1cbd064296f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.339782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084ca7f-8e04-48d1-9565-c1cbd064296f admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.356908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1097/2199] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:51.358245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67efea6-1134-451a-97b9-7d3a90f5a446 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.373518 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67efea6-1134-451a-97b9-7d3a90f5a446 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.382800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b052e-319f-43a8-8c24-e8139e59280d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.383825 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b052e-319f-43a8-8c24-e8139e59280d admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.384305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b052e-319f-43a8-8c24-e8139e59280d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.384779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b052e-319f-43a8-8c24-e8139e59280d admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.386029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b052e-319f-43a8-8c24-e8139e59280d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.392396 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b052e-319f-43a8-8c24-e8139e59280d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.501860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084ca7f-8e04-48d1-9565-c1cbd064296f admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.501860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a084ca7f-8e04-48d1-9565-c1cbd064296f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:51.512769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084ca7f-8e04-48d1-9565-c1cbd064296f admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.517925 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.523890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.523890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.523890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] PATH_INFO: `/v3/projects/39ac7f88ad47491f80705e0b006b3d57` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.523890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ac7f88ad47491f80705e0b006b3d57)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.523890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.538865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1103/2200] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:08:51 2022] POST /identity/v3/projects => generated 369 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:08:51.539953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b052e-319f-43a8-8c24-e8139e59280d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.557253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:51.611746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b052e-319f-43a8-8c24-e8139e59280d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.616582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61564a64-281f-4c38-ac24-60e736bec8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.617372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61564a64-281f-4c38-ac24-60e736bec8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.617578 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61564a64-281f-4c38-ac24-60e736bec8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.617859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61564a64-281f-4c38-ac24-60e736bec8c3 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.618638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61564a64-281f-4c38-ac24-60e736bec8c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.620569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61564a64-281f-4c38-ac24-60e736bec8c3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.681165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ac7f88ad47491f80705e0b006b3d57', 'request_id': 'req-895ef4f8-acef-44bd-beda-7834b227f86b'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:51.721597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1104/2201] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:08:51 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:08:51.725907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61564a64-281f-4c38-ac24-60e736bec8c3 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.736577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61564a64-281f-4c38-ac24-60e736bec8c3 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.795093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cd7a0-d1d3-4c6b-8612-64ceb6583b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.795823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cd7a0-d1d3-4c6b-8612-64ceb6583b89 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.796143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cd7a0-d1d3-4c6b-8612-64ceb6583b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.796422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cd7a0-d1d3-4c6b-8612-64ceb6583b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.797152 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809cd7a0-d1d3-4c6b-8612-64ceb6583b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.798875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809cd7a0-d1d3-4c6b-8612-64ceb6583b89 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.810266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1105/2202] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:08:51.814804 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cd7a0-d1d3-4c6b-8612-64ceb6583b89 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.825020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cd7a0-d1d3-4c6b-8612-64ceb6583b89 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.829181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ac7f88ad47491f80705e0b006b3d57', 'request_id': 'req-895ef4f8-acef-44bd-beda-7834b227f86b'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:51.831512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650825da-9734-482f-9a69-3e538d781129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.832238 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650825da-9734-482f-9a69-3e538d781129 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.832532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650825da-9734-482f-9a69-3e538d781129 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.832816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650825da-9734-482f-9a69-3e538d781129 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.833548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650825da-9734-482f-9a69-3e538d781129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.835200 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650825da-9734-482f-9a69-3e538d781129 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.836848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ac7f88ad47491f80705e0b006b3d57', 'request_id': 'req-895ef4f8-acef-44bd-beda-7834b227f86b'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:51.850519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1106/2203] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:08:51.850754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] The token cache is being invalidate because project 39ac7f88ad47491f80705e0b006b3d57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:08:51.850898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:51.851123 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650825da-9734-482f-9a69-3e538d781129 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.851563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:51.853397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1098/2204] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:08:51 2022] DELETE /identity/v3/projects/39ac7f88ad47491f80705e0b006b3d57 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:51.857327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.858916 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:51.862168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650825da-9734-482f-9a69-3e538d781129 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.866843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df7efd-6251-4979-ac13-12824359ff17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.879854 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df7efd-6251-4979-ac13-12824359ff17 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.879854 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df7efd-6251-4979-ac13-12824359ff17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.879854 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df7efd-6251-4979-ac13-12824359ff17 admin admin] PATH_INFO: `/v3/projects/bf9af35f313c48efba86e1a3ae0bc23c/users/96ff65becd4b47c69bb582b058e160ca/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.880203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ef4f8-acef-44bd-beda-7834b227f86b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.880203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fbb93-9d09-48e5-8c61-d3d00323e63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.880203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fbb93-9d09-48e5-8c61-d3d00323e63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.880203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fbb93-9d09-48e5-8c61-d3d00323e63e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.880577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fbb93-9d09-48e5-8c61-d3d00323e63e admin admin] PATH_INFO: `/v3/users/456ba76c09ee47d1b025f9ef419c3162` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.880764 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fbb93-9d09-48e5-8c61-d3d00323e63e admin admin] RBAC: Authorizing `identity:delete_user(user_id=456ba76c09ee47d1b025f9ef419c3162)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.882766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fbb93-9d09-48e5-8c61-d3d00323e63e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.885032 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df7efd-6251-4979-ac13-12824359ff17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf9af35f313c48efba86e1a3ae0bc23c, user_id=96ff65becd4b47c69bb582b058e160ca, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.887806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df7efd-6251-4979-ac13-12824359ff17 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.907731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:51.966121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1107/2205] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:08:51 2022] PUT /identity/v3/projects/bf9af35f313c48efba86e1a3ae0bc23c/users/96ff65becd4b47c69bb582b058e160ca/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:51.977509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268400e-5265-42fb-b871-49672960e4cc None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.977808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268400e-5265-42fb-b871-49672960e4cc None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.978097 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268400e-5265-42fb-b871-49672960e4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:52.011378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-9268400e-5265-42fb-b871-49672960e4cc None None] MFA Rules not processed for user `96ff65becd4b47c69bb582b058e160ca`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:52.012622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9268400e-5265-42fb-b871-49672960e4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:52.069304 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:52.079715 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1108/2206] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:08:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:08:52.091220 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268400e-5265-42fb-b871-49672960e4cc None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:52.092686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9268400e-5265-42fb-b871-49672960e4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:52.153076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e30fbb93-9d09-48e5-8c61-d3d00323e63e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '456ba76c09ee47d1b025f9ef419c3162'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:52.153606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e30fbb93-9d09-48e5-8c61-d3d00323e63e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '456ba76c09ee47d1b025f9ef419c3162'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:52.154426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e30fbb93-9d09-48e5-8c61-d3d00323e63e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '456ba76c09ee47d1b025f9ef419c3162'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:52.182356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268400e-5265-42fb-b871-49672960e4cc None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:52.186211 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81effaba-2ac4-43f3-9877-5c03513f3bc4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28954d67689a46ee975fdfc9139e41eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:52.186899 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81effaba-2ac4-43f3-9877-5c03513f3bc4 service glance] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:52.187212 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81effaba-2ac4-43f3-9877-5c03513f3bc4 service glance] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:52.187492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81effaba-2ac4-43f3-9877-5c03513f3bc4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:52.189397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-81effaba-2ac4-43f3-9877-5c03513f3bc4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:52.200842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81effaba-2ac4-43f3-9877-5c03513f3bc4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:52.202311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81effaba-2ac4-43f3-9877-5c03513f3bc4 service glance] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:52.211772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1109/2207] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:08:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:08:52.251429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e30fbb93-9d09-48e5-8c61-d3d00323e63e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '456ba76c09ee47d1b025f9ef419c3162'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:52.266837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e30fbb93-9d09-48e5-8c61-d3d00323e63e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '456ba76c09ee47d1b025f9ef419c3162'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:52.282427 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1099/2208] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:08:51 2022] DELETE /identity/v3/users/456ba76c09ee47d1b025f9ef419c3162 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:52.849016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81effaba-2ac4-43f3-9877-5c03513f3bc4 service glance] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:52.850784 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-81effaba-2ac4-43f3-9877-5c03513f3bc4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:52.935961 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81effaba-2ac4-43f3-9877-5c03513f3bc4 service glance] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:52.995525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:52.996318 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:52.996604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:52.996889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a admin admin] PATH_INFO: `/v3/projects/fc68f50149004e8a8bd939a31c078360` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:52.998087 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc68f50149004e8a8bd939a31c078360)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:53.000236 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:53.140542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:53.158918 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc68f50149004e8a8bd939a31c078360', 'request_id': 'req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:53.165408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc68f50149004e8a8bd939a31c078360', 'request_id': 'req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:53.174681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc68f50149004e8a8bd939a31c078360', 'request_id': 'req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:53.260629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a admin admin] The token cache is being invalidate because project fc68f50149004e8a8bd939a31c078360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:08:53.260629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:53.261054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-eb48c747-d31f-4fce-a661-0fe5e8bf8b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:08:53.263522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1110/2209] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:08:52 2022] DELETE /identity/v3/projects/fc68f50149004e8a8bd939a31c078360 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:53.269424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b5b0f-9b3f-4e39-b628-4709431c96d5 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.269961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b5b0f-9b3f-4e39-b628-4709431c96d5 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.269961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b5b0f-9b3f-4e39-b628-4709431c96d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.291477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3f4b5b0f-9b3f-4e39-b628-4709431c96d5 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:53.292703 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3f4b5b0f-9b3f-4e39-b628-4709431c96d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:53.357401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1100/2210] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:08:53.367239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370d62d-0b8e-4186-b053-5f62dc354474 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.371427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370d62d-0b8e-4186-b053-5f62dc354474 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.371427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370d62d-0b8e-4186-b053-5f62dc354474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.390380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-f370d62d-0b8e-4186-b053-5f62dc354474 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:53.391700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f370d62d-0b8e-4186-b053-5f62dc354474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:53.400415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1111/2211] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:08:53.404958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b5b0f-9b3f-4e39-b628-4709431c96d5 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:53.415826 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b5b0f-9b3f-4e39-b628-4709431c96d5 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.420728 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472869d2-355e-40fb-9911-3950904708a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:53.421506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472869d2-355e-40fb-9911-3950904708a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.421799 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472869d2-355e-40fb-9911-3950904708a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.422084 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472869d2-355e-40fb-9911-3950904708a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.423480 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472869d2-355e-40fb-9911-3950904708a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:53.423716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472869d2-355e-40fb-9911-3950904708a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:53.425498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472869d2-355e-40fb-9911-3950904708a6 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:53.439490 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1101/2212] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:53.443873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370d62d-0b8e-4186-b053-5f62dc354474 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:53.453535 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370d62d-0b8e-4186-b053-5f62dc354474 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.458879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ea131-b7be-4882-8cb2-8374184a11f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:53.459555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ea131-b7be-4882-8cb2-8374184a11f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.459844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ea131-b7be-4882-8cb2-8374184a11f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.460258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ea131-b7be-4882-8cb2-8374184a11f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.460874 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ea131-b7be-4882-8cb2-8374184a11f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:53.462812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ea131-b7be-4882-8cb2-8374184a11f5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:53.578214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1112/2213] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:08:53 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:08:53.583331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472869d2-355e-40fb-9911-3950904708a6 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:53.593718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472869d2-355e-40fb-9911-3950904708a6 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.654784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f51f6b-1f45-4211-b4e4-42cb42167d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:53.655468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f51f6b-1f45-4211-b4e4-42cb42167d70 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.655759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f51f6b-1f45-4211-b4e4-42cb42167d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.656042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f51f6b-1f45-4211-b4e4-42cb42167d70 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.657285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f51f6b-1f45-4211-b4e4-42cb42167d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:53.659557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f51f6b-1f45-4211-b4e4-42cb42167d70 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:53.743458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1102/2214] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:08:53 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:08:53.747723 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ea131-b7be-4882-8cb2-8374184a11f5 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:53.759700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ea131-b7be-4882-8cb2-8374184a11f5 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.764490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d43870-02eb-401d-a1c6-e9d5b1214ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:53.765150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d43870-02eb-401d-a1c6-e9d5b1214ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.765435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d43870-02eb-401d-a1c6-e9d5b1214ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.765711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d43870-02eb-401d-a1c6-e9d5b1214ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.767506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d43870-02eb-401d-a1c6-e9d5b1214ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:53.769261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d43870-02eb-401d-a1c6-e9d5b1214ea4 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:53.779887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1113/2215] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:08:53.785739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f51f6b-1f45-4211-b4e4-42cb42167d70 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:53.794729 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f51f6b-1f45-4211-b4e4-42cb42167d70 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.800664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec0e1c-25df-4f39-b4a2-1554eae56a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:53.801371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec0e1c-25df-4f39-b4a2-1554eae56a51 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.801660 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec0e1c-25df-4f39-b4a2-1554eae56a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.801944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec0e1c-25df-4f39-b4a2-1554eae56a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.802741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec0e1c-25df-4f39-b4a2-1554eae56a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:53.804394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec0e1c-25df-4f39-b4a2-1554eae56a51 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:53.817335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1103/2216] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:08:53.821733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d43870-02eb-401d-a1c6-e9d5b1214ea4 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:53.831925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d43870-02eb-401d-a1c6-e9d5b1214ea4 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.837284 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e1cb9-c4ab-41fb-9a99-5f7f1d781adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:53.838209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e1cb9-c4ab-41fb-9a99-5f7f1d781adf admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.838567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e1cb9-c4ab-41fb-9a99-5f7f1d781adf admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.838894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e1cb9-c4ab-41fb-9a99-5f7f1d781adf admin admin] PATH_INFO: `/v3/projects/2ac8207f8aa14d92a02ef75ee8b2fb8c/users/b170450d4ee84de7b48982d45f44236e/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.854662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e1cb9-c4ab-41fb-9a99-5f7f1d781adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac8207f8aa14d92a02ef75ee8b2fb8c, user_id=b170450d4ee84de7b48982d45f44236e, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:53.857367 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e1cb9-c4ab-41fb-9a99-5f7f1d781adf admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:53.931551 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1114/2217] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:08:53 2022] PUT /identity/v3/projects/2ac8207f8aa14d92a02ef75ee8b2fb8c/users/b170450d4ee84de7b48982d45f44236e/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:53.942929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041bf4a-0ec1-429a-89bc-b1bdf69cd421 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.943220 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041bf4a-0ec1-429a-89bc-b1bdf69cd421 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.943499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041bf4a-0ec1-429a-89bc-b1bdf69cd421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.952097 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:53.972146 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-6041bf4a-0ec1-429a-89bc-b1bdf69cd421 None None] MFA Rules not processed for user `b170450d4ee84de7b48982d45f44236e`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:53.973403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6041bf4a-0ec1-429a-89bc-b1bdf69cd421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:54.043069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1104/2218] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:08:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:08:54.054534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e1cb9-c4ab-41fb-9a99-5f7f1d781adf admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:54.055989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1c0e1cb9-c4ab-41fb-9a99-5f7f1d781adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:54.118103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e1cb9-c4ab-41fb-9a99-5f7f1d781adf admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:54.121869 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543ffa8-557a-4750-bf92-11c4fb87f1dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:54.122562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543ffa8-557a-4750-bf92-11c4fb87f1dd service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:54.123095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543ffa8-557a-4750-bf92-11c4fb87f1dd service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:54.123380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543ffa8-557a-4750-bf92-11c4fb87f1dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:54.133901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543ffa8-557a-4750-bf92-11c4fb87f1dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:54.135396 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543ffa8-557a-4750-bf92-11c4fb87f1dd service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:54.146502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1115/2219] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:08:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:09:04.068188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=30 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:04.141482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:12.317283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041bf4a-0ec1-429a-89bc-b1bdf69cd421 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:12.402610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041bf4a-0ec1-429a-89bc-b1bdf69cd421 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:12.408219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:12.409128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:12.409509 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:12.409748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] PATH_INFO: `/v3/users/b170450d4ee84de7b48982d45f44236e` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:12.411057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b170450d4ee84de7b48982d45f44236e)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:12.413116 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:12.591973 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b170450d4ee84de7b48982d45f44236e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:12.592671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b170450d4ee84de7b48982d45f44236e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:12.593282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b170450d4ee84de7b48982d45f44236e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:12.688983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b170450d4ee84de7b48982d45f44236e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:12.714226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b170450d4ee84de7b48982d45f44236e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:12.736188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1105/2220] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:09:12 2022] DELETE /identity/v3/users/b170450d4ee84de7b48982d45f44236e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:12.741577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc96304-2552-4721-ba84-718a4c010c8e None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:12.742085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc96304-2552-4721-ba84-718a4c010c8e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:12.742199 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc96304-2552-4721-ba84-718a4c010c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:12.773774 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-4bc96304-2552-4721-ba84-718a4c010c8e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:12.776644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-4bc96304-2552-4721-ba84-718a4c010c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:12.863748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1116/2221] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:09:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:09:12.876305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:12.878779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:12.960707 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398c99c-dde2-47d8-9301-8f49fc3a2417 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:12.965542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdb2af-363c-4010-9561-8fe837831182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:12.966397 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdb2af-363c-4010-9561-8fe837831182 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:12.966652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdb2af-363c-4010-9561-8fe837831182 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:12.966937 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdb2af-363c-4010-9561-8fe837831182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:12.980644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdb2af-363c-4010-9561-8fe837831182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:12.983133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdb2af-363c-4010-9561-8fe837831182 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:13.000563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1106/2222] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:09:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:09:13.619734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc96304-2552-4721-ba84-718a4c010c8e None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:13.621415 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-4bc96304-2552-4721-ba84-718a4c010c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:13.650868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc96304-2552-4721-ba84-718a4c010c8e None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:13.654652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:13.655960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:13.657016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:13.657595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf admin admin] PATH_INFO: `/v3/projects/2ac8207f8aa14d92a02ef75ee8b2fb8c` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:13.660945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac8207f8aa14d92a02ef75ee8b2fb8c)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:13.664472 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:13.829295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac8207f8aa14d92a02ef75ee8b2fb8c', 'request_id': 'req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:13.846996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac8207f8aa14d92a02ef75ee8b2fb8c', 'request_id': 'req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:13.846996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac8207f8aa14d92a02ef75ee8b2fb8c', 'request_id': 'req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:13.972764 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf admin admin] The token cache is being invalidate because project 2ac8207f8aa14d92a02ef75ee8b2fb8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:09:13.973324 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:13.974022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1bfeaaef-6045-45c8-a00d-a347c57a3eaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:13.977267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1117/2223] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:09:13 2022] DELETE /identity/v3/projects/2ac8207f8aa14d92a02ef75ee8b2fb8c => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:13.983003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397f69d-a63e-453a-8173-988fc841ea9e None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:13.983546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397f69d-a63e-453a-8173-988fc841ea9e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:13.983546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397f69d-a63e-453a-8173-988fc841ea9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.010894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3397f69d-a63e-453a-8173-988fc841ea9e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:14.012539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3397f69d-a63e-453a-8173-988fc841ea9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:14.032298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:14.085479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1107/2224] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:09:14.096172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238933b-763a-40f9-9a51-73fcac32495e None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.096486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238933b-763a-40f9-9a51-73fcac32495e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.096775 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238933b-763a-40f9-9a51-73fcac32495e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.127765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-c238933b-763a-40f9-9a51-73fcac32495e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:14.129380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c238933b-763a-40f9-9a51-73fcac32495e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:14.146977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1118/2225] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:09:14.154116 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397f69d-a63e-453a-8173-988fc841ea9e None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:14.174477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397f69d-a63e-453a-8173-988fc841ea9e None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:14.181485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b47069-4aab-4c8f-9bb0-e5456cf92913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:14.182727 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b47069-4aab-4c8f-9bb0-e5456cf92913 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.183131 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b47069-4aab-4c8f-9bb0-e5456cf92913 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.183655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b47069-4aab-4c8f-9bb0-e5456cf92913 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.184328 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b47069-4aab-4c8f-9bb0-e5456cf92913 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:14.185457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b47069-4aab-4c8f-9bb0-e5456cf92913 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:14.187333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b47069-4aab-4c8f-9bb0-e5456cf92913 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:14.206032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1108/2226] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:14.215044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238933b-763a-40f9-9a51-73fcac32495e None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:14.226239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238933b-763a-40f9-9a51-73fcac32495e None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:14.231346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac7ae6-422f-4ed1-9e01-b4eaf3b2df5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:14.232022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac7ae6-422f-4ed1-9e01-b4eaf3b2df5a admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.232396 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac7ae6-422f-4ed1-9e01-b4eaf3b2df5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.232693 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac7ae6-422f-4ed1-9e01-b4eaf3b2df5a admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.233517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac7ae6-422f-4ed1-9e01-b4eaf3b2df5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:14.236425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac7ae6-422f-4ed1-9e01-b4eaf3b2df5a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:14.356624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1119/2227] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:09:14 2022] POST /identity/v3/projects => generated 361 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:14.362400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b47069-4aab-4c8f-9bb0-e5456cf92913 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:14.372974 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b47069-4aab-4c8f-9bb0-e5456cf92913 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:14.445480 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f280a-21ee-4b49-81dd-e7fb9a0062fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:14.446310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f280a-21ee-4b49-81dd-e7fb9a0062fc admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.446634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f280a-21ee-4b49-81dd-e7fb9a0062fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.446918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f280a-21ee-4b49-81dd-e7fb9a0062fc admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.447644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f280a-21ee-4b49-81dd-e7fb9a0062fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:14.449850 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f280a-21ee-4b49-81dd-e7fb9a0062fc admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:14.586672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1109/2228] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:09:14 2022] POST /identity/v3/users => generated 281 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:14.607414 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac7ae6-422f-4ed1-9e01-b4eaf3b2df5a admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:14.622070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac7ae6-422f-4ed1-9e01-b4eaf3b2df5a admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:14.629101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafa4fd-5eef-4994-bbbe-5d54be5ea094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:14.630372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafa4fd-5eef-4994-bbbe-5d54be5ea094 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.630665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafa4fd-5eef-4994-bbbe-5d54be5ea094 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.630944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafa4fd-5eef-4994-bbbe-5d54be5ea094 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.631714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafa4fd-5eef-4994-bbbe-5d54be5ea094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:14.633438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafa4fd-5eef-4994-bbbe-5d54be5ea094 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:14.647179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1120/2229] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:09:14.653255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f280a-21ee-4b49-81dd-e7fb9a0062fc admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:14.667141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f280a-21ee-4b49-81dd-e7fb9a0062fc admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:14.673302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147bad1-b7df-427f-bd32-1fc66fbbc8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:14.674072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147bad1-b7df-427f-bd32-1fc66fbbc8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.674313 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147bad1-b7df-427f-bd32-1fc66fbbc8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.674566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147bad1-b7df-427f-bd32-1fc66fbbc8f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.675306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147bad1-b7df-427f-bd32-1fc66fbbc8f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:14.676968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147bad1-b7df-427f-bd32-1fc66fbbc8f3 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:14.690024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1110/2230] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:09:14.695675 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafa4fd-5eef-4994-bbbe-5d54be5ea094 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:14.708927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafa4fd-5eef-4994-bbbe-5d54be5ea094 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:14.714298 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969aebe4-ac0f-410a-9877-9f96710005cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:14.715313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969aebe4-ac0f-410a-9877-9f96710005cd admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.715609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969aebe4-ac0f-410a-9877-9f96710005cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.715895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969aebe4-ac0f-410a-9877-9f96710005cd admin admin] PATH_INFO: `/v3/projects/b0e81a0abbda4e879caa5b8e1d20781d/users/9c73aa7a19ca4b5da2fd6171e4ef68d1/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.732960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969aebe4-ac0f-410a-9877-9f96710005cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e81a0abbda4e879caa5b8e1d20781d, user_id=9c73aa7a19ca4b5da2fd6171e4ef68d1, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:14.736811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969aebe4-ac0f-410a-9877-9f96710005cd admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:14.810461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1121/2231] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:09:14 2022] PUT /identity/v3/projects/b0e81a0abbda4e879caa5b8e1d20781d/users/9c73aa7a19ca4b5da2fd6171e4ef68d1/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:14.823013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc6f8d-3f46-4fad-8d8e-fd4e4fc5c6e8 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.823314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc6f8d-3f46-4fad-8d8e-fd4e4fc5c6e8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.826884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc6f8d-3f46-4fad-8d8e-fd4e4fc5c6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.859864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-2dcc6f8d-3f46-4fad-8d8e-fd4e4fc5c6e8 None None] MFA Rules not processed for user `9c73aa7a19ca4b5da2fd6171e4ef68d1`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:14.861266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2dcc6f8d-3f46-4fad-8d8e-fd4e4fc5c6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:14.959630 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1111/2232] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:09:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:09:14.997935 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969aebe4-ac0f-410a-9877-9f96710005cd admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:15.001152 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-969aebe4-ac0f-410a-9877-9f96710005cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:15.032131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:15.089775 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969aebe4-ac0f-410a-9877-9f96710005cd admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:15.094962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e3d14-dbc9-4400-a17d-579dbf6034d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:15.095637 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e3d14-dbc9-4400-a17d-579dbf6034d9 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:15.095921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e3d14-dbc9-4400-a17d-579dbf6034d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:15.096198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e3d14-dbc9-4400-a17d-579dbf6034d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:15.105575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e3d14-dbc9-4400-a17d-579dbf6034d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:15.107051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e3d14-dbc9-4400-a17d-579dbf6034d9 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:15.120159 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1122/2233] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:09:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:09:15.137208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc6f8d-3f46-4fad-8d8e-fd4e4fc5c6e8 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:15.172572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:15.237676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc6f8d-3f46-4fad-8d8e-fd4e4fc5c6e8 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:15.243043 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:15.244164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:15.244836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:15.245030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] PATH_INFO: `/v3/users/9c73aa7a19ca4b5da2fd6171e4ef68d1` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:15.246400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c73aa7a19ca4b5da2fd6171e4ef68d1)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:15.250693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:15.492245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c73aa7a19ca4b5da2fd6171e4ef68d1'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:15.493241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c73aa7a19ca4b5da2fd6171e4ef68d1'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:15.493894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c73aa7a19ca4b5da2fd6171e4ef68d1'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:15.564061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c73aa7a19ca4b5da2fd6171e4ef68d1'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:15.582217 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c73aa7a19ca4b5da2fd6171e4ef68d1'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:15.605811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1112/2234] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:09:15 2022] DELETE /identity/v3/users/9c73aa7a19ca4b5da2fd6171e4ef68d1 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:15.611724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4df77-13f4-4a1c-b0ae-337e48dec082 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:15.612047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4df77-13f4-4a1c-b0ae-337e48dec082 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:15.612388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4df77-13f4-4a1c-b0ae-337e48dec082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:15.635565 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-cda4df77-13f4-4a1c-b0ae-337e48dec082 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:15.636842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-cda4df77-13f4-4a1c-b0ae-337e48dec082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:15.737587 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1123/2235] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:09:15.748513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:15.750045 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:15.841547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571b1c0-e197-4d4c-9862-f666fbdd7949 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:15.846511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c073a-1f3e-475e-a371-4ccea240a2be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:15.847196 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c073a-1f3e-475e-a371-4ccea240a2be service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:15.847478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c073a-1f3e-475e-a371-4ccea240a2be service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:15.847762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c073a-1f3e-475e-a371-4ccea240a2be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:15.858572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c073a-1f3e-475e-a371-4ccea240a2be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:15.860610 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c073a-1f3e-475e-a371-4ccea240a2be service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:15.872313 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1113/2236] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:09:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:09:16.373085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4df77-13f4-4a1c-b0ae-337e48dec082 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:16.379064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-cda4df77-13f4-4a1c-b0ae-337e48dec082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:16.391414 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4df77-13f4-4a1c-b0ae-337e48dec082 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:16.398089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a4d35-44e7-471c-b49e-e04ae5da7370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:16.399202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a4d35-44e7-471c-b49e-e04ae5da7370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:16.399639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a4d35-44e7-471c-b49e-e04ae5da7370 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:16.400064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a4d35-44e7-471c-b49e-e04ae5da7370 admin admin] PATH_INFO: `/v3/projects/b0e81a0abbda4e879caa5b8e1d20781d` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:16.402020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a4d35-44e7-471c-b49e-e04ae5da7370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0e81a0abbda4e879caa5b8e1d20781d)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:16.405265 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a4d35-44e7-471c-b49e-e04ae5da7370 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:16.616922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-195a4d35-44e7-471c-b49e-e04ae5da7370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0e81a0abbda4e879caa5b8e1d20781d', 'request_id': 'req-195a4d35-44e7-471c-b49e-e04ae5da7370'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:16.623250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-195a4d35-44e7-471c-b49e-e04ae5da7370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0e81a0abbda4e879caa5b8e1d20781d', 'request_id': 'req-195a4d35-44e7-471c-b49e-e04ae5da7370'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:16.629858 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-195a4d35-44e7-471c-b49e-e04ae5da7370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0e81a0abbda4e879caa5b8e1d20781d', 'request_id': 'req-195a4d35-44e7-471c-b49e-e04ae5da7370'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:16.718583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-195a4d35-44e7-471c-b49e-e04ae5da7370 admin admin] The token cache is being invalidate because project b0e81a0abbda4e879caa5b8e1d20781d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:09:16.719655 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-195a4d35-44e7-471c-b49e-e04ae5da7370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:16.720451 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-195a4d35-44e7-471c-b49e-e04ae5da7370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:16.725623 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1124/2237] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:09:16 2022] DELETE /identity/v3/projects/b0e81a0abbda4e879caa5b8e1d20781d => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:16.730240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b36c7-15ee-464b-88df-d895bb6de33b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:16.730752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b36c7-15ee-464b-88df-d895bb6de33b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:16.730835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b36c7-15ee-464b-88df-d895bb6de33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:16.755629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-9a4b36c7-15ee-464b-88df-d895bb6de33b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:16.759549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9a4b36c7-15ee-464b-88df-d895bb6de33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:16.793330 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:16.822461 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1114/2238] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:09:16.830461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240614d-3241-4ed8-9862-7d0a28a6c2f7 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:16.832245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240614d-3241-4ed8-9862-7d0a28a6c2f7 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:16.832245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240614d-3241-4ed8-9862-7d0a28a6c2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:16.854142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-5240614d-3241-4ed8-9862-7d0a28a6c2f7 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:16.854142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5240614d-3241-4ed8-9862-7d0a28a6c2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:16.864033 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1125/2239] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:09:16.874208 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b36c7-15ee-464b-88df-d895bb6de33b None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:16.885737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b36c7-15ee-464b-88df-d895bb6de33b None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:16.893339 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc412a6-2c36-4eab-9d84-99cd9a395215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:16.894552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc412a6-2c36-4eab-9d84-99cd9a395215 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:16.895081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc412a6-2c36-4eab-9d84-99cd9a395215 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:16.895571 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc412a6-2c36-4eab-9d84-99cd9a395215 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:16.897150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc412a6-2c36-4eab-9d84-99cd9a395215 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:16.897726 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc412a6-2c36-4eab-9d84-99cd9a395215 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:16.901257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc412a6-2c36-4eab-9d84-99cd9a395215 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:16.917205 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1115/2240] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:16.930669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240614d-3241-4ed8-9862-7d0a28a6c2f7 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:16.944434 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240614d-3241-4ed8-9862-7d0a28a6c2f7 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:16.949900 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeef438-24a2-45ae-b33b-8aaffa8b12a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:16.950709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeef438-24a2-45ae-b33b-8aaffa8b12a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:16.950999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeef438-24a2-45ae-b33b-8aaffa8b12a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:16.951480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeef438-24a2-45ae-b33b-8aaffa8b12a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:16.952203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeef438-24a2-45ae-b33b-8aaffa8b12a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:16.954241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeef438-24a2-45ae-b33b-8aaffa8b12a3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:17.073115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1126/2241] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:09:16 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:09:17.077510 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc412a6-2c36-4eab-9d84-99cd9a395215 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:17.089404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc412a6-2c36-4eab-9d84-99cd9a395215 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:17.160134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bc0d0-555b-47be-b51a-5937e6d1337e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:17.160738 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bc0d0-555b-47be-b51a-5937e6d1337e admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:17.161021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bc0d0-555b-47be-b51a-5937e6d1337e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:17.161301 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bc0d0-555b-47be-b51a-5937e6d1337e admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:17.162003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bc0d0-555b-47be-b51a-5937e6d1337e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:17.164305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bc0d0-555b-47be-b51a-5937e6d1337e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:17.343019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1116/2242] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:09:17 2022] POST /identity/v3/users => generated 287 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:17.348462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeef438-24a2-45ae-b33b-8aaffa8b12a3 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:17.360115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeef438-24a2-45ae-b33b-8aaffa8b12a3 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:17.373356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c3ca0-4b18-4432-9c87-fd8013c8798a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:17.373356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c3ca0-4b18-4432-9c87-fd8013c8798a admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:17.373356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c3ca0-4b18-4432-9c87-fd8013c8798a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:17.373356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c3ca0-4b18-4432-9c87-fd8013c8798a admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:17.374230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c3ca0-4b18-4432-9c87-fd8013c8798a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:17.374659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c3ca0-4b18-4432-9c87-fd8013c8798a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:17.392572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1127/2243] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:09:17.396741 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bc0d0-555b-47be-b51a-5937e6d1337e admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:17.409588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bc0d0-555b-47be-b51a-5937e6d1337e admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:17.418926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba79638-6835-4479-8afe-ec9bce13e406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:17.419551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba79638-6835-4479-8afe-ec9bce13e406 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:17.419830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba79638-6835-4479-8afe-ec9bce13e406 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:17.420113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba79638-6835-4479-8afe-ec9bce13e406 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:17.420981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba79638-6835-4479-8afe-ec9bce13e406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:17.423835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba79638-6835-4479-8afe-ec9bce13e406 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:17.444946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1117/2244] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:09:17.485066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c3ca0-4b18-4432-9c87-fd8013c8798a admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:17.496117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c3ca0-4b18-4432-9c87-fd8013c8798a admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:17.502209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dc29f-8e7b-48f1-8e52-a3273a6c716e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:17.503264 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dc29f-8e7b-48f1-8e52-a3273a6c716e admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:17.503619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dc29f-8e7b-48f1-8e52-a3273a6c716e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:17.503986 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dc29f-8e7b-48f1-8e52-a3273a6c716e admin admin] PATH_INFO: `/v3/projects/1e78a81da7e645b784edd4823b4a271e/users/c740361592354dd39b41a264e250fd4b/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:17.507541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:17.529814 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dc29f-8e7b-48f1-8e52-a3273a6c716e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e78a81da7e645b784edd4823b4a271e, user_id=c740361592354dd39b41a264e250fd4b, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:17.533088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dc29f-8e7b-48f1-8e52-a3273a6c716e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:17.638606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1128/2245] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:09:17 2022] PUT /identity/v3/projects/1e78a81da7e645b784edd4823b4a271e/users/c740361592354dd39b41a264e250fd4b/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:17.653005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff262288-7315-482a-9009-c0922eea2832 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:17.653297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff262288-7315-482a-9009-c0922eea2832 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:17.653588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff262288-7315-482a-9009-c0922eea2832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:17.688335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ff262288-7315-482a-9009-c0922eea2832 None None] MFA Rules not processed for user `c740361592354dd39b41a264e250fd4b`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:17.694269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ff262288-7315-482a-9009-c0922eea2832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:17.772937 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1118/2246] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:09:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:09:17.783661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dc29f-8e7b-48f1-8e52-a3273a6c716e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:17.888949 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dc29f-8e7b-48f1-8e52-a3273a6c716e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:17.894872 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadec4c9-731a-4ac3-a4c9-848d7eb1ee3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:17.895559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadec4c9-731a-4ac3-a4c9-848d7eb1ee3b admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:17.895855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadec4c9-731a-4ac3-a4c9-848d7eb1ee3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:17.896148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadec4c9-731a-4ac3-a4c9-848d7eb1ee3b admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:17.896914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadec4c9-731a-4ac3-a4c9-848d7eb1ee3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:17.899070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadec4c9-731a-4ac3-a4c9-848d7eb1ee3b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:18.026777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1129/2247] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:09:17 2022] POST /identity/v3/projects => generated 375 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:09:18.030728 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff262288-7315-482a-9009-c0922eea2832 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:18.044591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff262288-7315-482a-9009-c0922eea2832 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:18.104101 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:18.149089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a30fc7-ed64-4791-ae5b-c994322810e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:18.149851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a30fc7-ed64-4791-ae5b-c994322810e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:18.150230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a30fc7-ed64-4791-ae5b-c994322810e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:18.150544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a30fc7-ed64-4791-ae5b-c994322810e3 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:18.151267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a30fc7-ed64-4791-ae5b-c994322810e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:18.153217 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a30fc7-ed64-4791-ae5b-c994322810e3 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:18.275387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1119/2248] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:09:18 2022] POST /identity/v3/users => generated 294 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:18.278660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadec4c9-731a-4ac3-a4c9-848d7eb1ee3b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:18.294560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadec4c9-731a-4ac3-a4c9-848d7eb1ee3b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:18.300413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b62f04-f4c4-4ce1-a2b3-a0dfe342bea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:18.301424 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b62f04-f4c4-4ce1-a2b3-a0dfe342bea5 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:18.301855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b62f04-f4c4-4ce1-a2b3-a0dfe342bea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:18.302294 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b62f04-f4c4-4ce1-a2b3-a0dfe342bea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:18.303601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b62f04-f4c4-4ce1-a2b3-a0dfe342bea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:18.306586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b62f04-f4c4-4ce1-a2b3-a0dfe342bea5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:18.319911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1130/2249] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:09:18.325008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a30fc7-ed64-4791-ae5b-c994322810e3 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:18.339831 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a30fc7-ed64-4791-ae5b-c994322810e3 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:18.350536 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08514708-07f4-4dc9-9d66-074a8aeb6a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:18.350536 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08514708-07f4-4dc9-9d66-074a8aeb6a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:18.351823 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08514708-07f4-4dc9-9d66-074a8aeb6a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:18.351823 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08514708-07f4-4dc9-9d66-074a8aeb6a3b admin admin] PATH_INFO: `/v3/projects/7f3af8781d2348d7b6792cd3e40324f9/users/e96577b684df4fc08e66c76b06da2000/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:18.387209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08514708-07f4-4dc9-9d66-074a8aeb6a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f3af8781d2348d7b6792cd3e40324f9, user_id=e96577b684df4fc08e66c76b06da2000, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:18.393407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08514708-07f4-4dc9-9d66-074a8aeb6a3b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:18.509702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1120/2250] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:09:18 2022] PUT /identity/v3/projects/7f3af8781d2348d7b6792cd3e40324f9/users/e96577b684df4fc08e66c76b06da2000/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:18.513760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b62f04-f4c4-4ce1-a2b3-a0dfe342bea5 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:18.524444 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b62f04-f4c4-4ce1-a2b3-a0dfe342bea5 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:18.602533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc55880-6c66-4512-961a-109471443e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:18.604016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc55880-6c66-4512-961a-109471443e71 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:18.604016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc55880-6c66-4512-961a-109471443e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:18.604016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc55880-6c66-4512-961a-109471443e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:18.604597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc55880-6c66-4512-961a-109471443e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:18.606440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc55880-6c66-4512-961a-109471443e71 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:18.623098 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1131/2251] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:09:18.632224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08514708-07f4-4dc9-9d66-074a8aeb6a3b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:18.644038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08514708-07f4-4dc9-9d66-074a8aeb6a3b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:18.650911 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f9fee-9971-492e-bcd9-9555551232bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:18.652001 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f9fee-9971-492e-bcd9-9555551232bc admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:18.652379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f9fee-9971-492e-bcd9-9555551232bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:18.652753 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f9fee-9971-492e-bcd9-9555551232bc admin admin] PATH_INFO: `/v3/projects/7f3af8781d2348d7b6792cd3e40324f9/users/e96577b684df4fc08e66c76b06da2000/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:18.655105 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f9fee-9971-492e-bcd9-9555551232bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f3af8781d2348d7b6792cd3e40324f9, user_id=e96577b684df4fc08e66c76b06da2000, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:18.658722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f9fee-9971-492e-bcd9-9555551232bc admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:18.670439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1121/2252] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:09:18 2022] PUT /identity/v3/projects/7f3af8781d2348d7b6792cd3e40324f9/users/e96577b684df4fc08e66c76b06da2000/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:18.681949 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b7b21-85a6-4efe-92b1-8eae2ac11221 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:18.682384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b7b21-85a6-4efe-92b1-8eae2ac11221 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:18.682749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b7b21-85a6-4efe-92b1-8eae2ac11221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:18.717473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-096b7b21-85a6-4efe-92b1-8eae2ac11221 None None] MFA Rules not processed for user `e96577b684df4fc08e66c76b06da2000`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:18.719171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-096b7b21-85a6-4efe-92b1-8eae2ac11221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:18.777793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:18.809010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1132/2253] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:09:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:09:18.821681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f9fee-9971-492e-bcd9-9555551232bc admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:18.823637 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-509f9fee-9971-492e-bcd9-9555551232bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:18.904913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f9fee-9971-492e-bcd9-9555551232bc admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:18.909353 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39528e9-7c72-4497-9fd1-2d3111c18b21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:18.910042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39528e9-7c72-4497-9fd1-2d3111c18b21 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:18.910359 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39528e9-7c72-4497-9fd1-2d3111c18b21 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:18.910644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39528e9-7c72-4497-9fd1-2d3111c18b21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:18.980030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39528e9-7c72-4497-9fd1-2d3111c18b21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:18.981579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39528e9-7c72-4497-9fd1-2d3111c18b21 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:18.997868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1122/2254] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:09:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:09:28.884334 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=10 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:28.990662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:29.802020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b7b21-85a6-4efe-92b1-8eae2ac11221 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:29.803487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-096b7b21-85a6-4efe-92b1-8eae2ac11221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:29.900763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b7b21-85a6-4efe-92b1-8eae2ac11221 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:29.908411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c768c-3a85-4d8c-a0ee-575cf1341207 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:29.909079 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c768c-3a85-4d8c-a0ee-575cf1341207 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:29.909481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c768c-3a85-4d8c-a0ee-575cf1341207 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:29.909771 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c768c-3a85-4d8c-a0ee-575cf1341207 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:29.911835 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-2c8c768c-3a85-4d8c-a0ee-575cf1341207 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:29.975414 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c768c-3a85-4d8c-a0ee-575cf1341207 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:29.977044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c768c-3a85-4d8c-a0ee-575cf1341207 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:29.987259 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1133/2255] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:09:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:09:39.982049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=56 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:41.160673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39528e9-7c72-4497-9fd1-2d3111c18b21 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:41.247504 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39528e9-7c72-4497-9fd1-2d3111c18b21 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:41.254834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:41.255797 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:41.256158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:41.256515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] PATH_INFO: `/v3/users/c740361592354dd39b41a264e250fd4b` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:41.257859 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c740361592354dd39b41a264e250fd4b)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:41.260347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:41.521129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c740361592354dd39b41a264e250fd4b'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:41.521734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c740361592354dd39b41a264e250fd4b'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:41.522435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c740361592354dd39b41a264e250fd4b'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:41.594640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c740361592354dd39b41a264e250fd4b'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:41.608249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c740361592354dd39b41a264e250fd4b'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:41.621735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1123/2256] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:09:41 2022] DELETE /identity/v3/users/c740361592354dd39b41a264e250fd4b => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:41.627100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03759e-b233-4993-b7fb-a916ef38f4d9 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:41.630292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03759e-b233-4993-b7fb-a916ef38f4d9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:41.630292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03759e-b233-4993-b7fb-a916ef38f4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:41.679058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-8b03759e-b233-4993-b7fb-a916ef38f4d9 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:41.681136 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8b03759e-b233-4993-b7fb-a916ef38f4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:41.770437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1134/2257] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:09:41.781070 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:41.783368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:41.977070 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149a80d-fb68-4b43-b340-bff0fee9c7cc admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:41.980804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4677eae-7a09-45b7-8697-b776d9a724e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:41.981468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4677eae-7a09-45b7-8697-b776d9a724e2 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:41.981754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4677eae-7a09-45b7-8697-b776d9a724e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:41.982223 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4677eae-7a09-45b7-8697-b776d9a724e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:41.995896 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4677eae-7a09-45b7-8697-b776d9a724e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:41.997331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4677eae-7a09-45b7-8697-b776d9a724e2 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:42.009598 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1124/2258] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:09:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:09:42.535916 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03759e-b233-4993-b7fb-a916ef38f4d9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:42.539858 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8b03759e-b233-4993-b7fb-a916ef38f4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:42.555359 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03759e-b233-4993-b7fb-a916ef38f4d9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:42.564901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:42.565802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:42.566188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:42.566674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] PATH_INFO: `/v3/projects/1e78a81da7e645b784edd4823b4a271e` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:42.567646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e78a81da7e645b784edd4823b4a271e)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:42.569601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:42.679430 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e78a81da7e645b784edd4823b4a271e', 'request_id': 'req-803f0edf-6154-42b4-873d-9a2136277ba7'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:42.686572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e78a81da7e645b784edd4823b4a271e', 'request_id': 'req-803f0edf-6154-42b4-873d-9a2136277ba7'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:42.693511 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e78a81da7e645b784edd4823b4a271e', 'request_id': 'req-803f0edf-6154-42b4-873d-9a2136277ba7'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:42.778343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] The token cache is being invalidate because project 1e78a81da7e645b784edd4823b4a271e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:09:42.778674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:42.779364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:42.781534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1135/2259] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:09:42 2022] DELETE /identity/v3/projects/1e78a81da7e645b784edd4823b4a271e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:42.785714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4677eae-7a09-45b7-8697-b776d9a724e2 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:42.796816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4677eae-7a09-45b7-8697-b776d9a724e2 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:42.818756 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:42.858417 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82abe4-7005-44c0-8419-0a024a63f4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:42.859340 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82abe4-7005-44c0-8419-0a024a63f4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:42.859702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82abe4-7005-44c0-8419-0a024a63f4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:42.859908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82abe4-7005-44c0-8419-0a024a63f4ee admin admin] PATH_INFO: `/v3/users/e96577b684df4fc08e66c76b06da2000` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:42.861187 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82abe4-7005-44c0-8419-0a024a63f4ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e96577b684df4fc08e66c76b06da2000)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:42.863246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82abe4-7005-44c0-8419-0a024a63f4ee admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:43.078519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-3c82abe4-7005-44c0-8419-0a024a63f4ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e96577b684df4fc08e66c76b06da2000'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:43.079108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-3c82abe4-7005-44c0-8419-0a024a63f4ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e96577b684df4fc08e66c76b06da2000'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:43.079720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-3c82abe4-7005-44c0-8419-0a024a63f4ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e96577b684df4fc08e66c76b06da2000'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:43.168685 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-3c82abe4-7005-44c0-8419-0a024a63f4ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e96577b684df4fc08e66c76b06da2000'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:43.182533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-3c82abe4-7005-44c0-8419-0a024a63f4ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e96577b684df4fc08e66c76b06da2000'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:43.196162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1125/2260] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:09:42 2022] DELETE /identity/v3/users/e96577b684df4fc08e66c76b06da2000 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:43.646067 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:43.647786 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:43.662319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f0edf-6154-42b4-873d-9a2136277ba7 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:43.722064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef0f0e-b3cf-488f-8359-488b463279e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:43.722969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef0f0e-b3cf-488f-8359-488b463279e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:43.723264 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef0f0e-b3cf-488f-8359-488b463279e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:43.723617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef0f0e-b3cf-488f-8359-488b463279e3 admin admin] PATH_INFO: `/v3/projects/7f3af8781d2348d7b6792cd3e40324f9` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:43.724755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef0f0e-b3cf-488f-8359-488b463279e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f3af8781d2348d7b6792cd3e40324f9)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:43.726718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef0f0e-b3cf-488f-8359-488b463279e3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:43.836085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:43.855214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-89ef0f0e-b3cf-488f-8359-488b463279e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f3af8781d2348d7b6792cd3e40324f9', 'request_id': 'req-89ef0f0e-b3cf-488f-8359-488b463279e3'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:43.866130 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-89ef0f0e-b3cf-488f-8359-488b463279e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f3af8781d2348d7b6792cd3e40324f9', 'request_id': 'req-89ef0f0e-b3cf-488f-8359-488b463279e3'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:43.873081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-89ef0f0e-b3cf-488f-8359-488b463279e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f3af8781d2348d7b6792cd3e40324f9', 'request_id': 'req-89ef0f0e-b3cf-488f-8359-488b463279e3'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:43.942435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-89ef0f0e-b3cf-488f-8359-488b463279e3 admin admin] The token cache is being invalidate because project 7f3af8781d2348d7b6792cd3e40324f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:09:43.942717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-89ef0f0e-b3cf-488f-8359-488b463279e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:43.943376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-89ef0f0e-b3cf-488f-8359-488b463279e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:09:43.944950 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1136/2261] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:09:43 2022] DELETE /identity/v3/projects/7f3af8781d2348d7b6792cd3e40324f9 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:43.949892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6dcc2-ec5f-4605-8561-d3b3a8d3b5d1 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:43.957898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6dcc2-ec5f-4605-8561-d3b3a8d3b5d1 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:43.957898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6dcc2-ec5f-4605-8561-d3b3a8d3b5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:43.973296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b2e6dcc2-ec5f-4605-8561-d3b3a8d3b5d1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:43.974572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b2e6dcc2-ec5f-4605-8561-d3b3a8d3b5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:44.040064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1126/2262] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:09:44.048171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372396fc-43de-460e-ae2f-80f40b376d75 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:44.048474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372396fc-43de-460e-ae2f-80f40b376d75 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:44.048757 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372396fc-43de-460e-ae2f-80f40b376d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:44.069862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-372396fc-43de-460e-ae2f-80f40b376d75 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:44.071120 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-372396fc-43de-460e-ae2f-80f40b376d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:44.080421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1137/2263] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:09:44.084802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6dcc2-ec5f-4605-8561-d3b3a8d3b5d1 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:44.095637 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6dcc2-ec5f-4605-8561-d3b3a8d3b5d1 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:44.100351 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca9160-e752-472c-bcef-5e63778a3955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:44.101014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca9160-e752-472c-bcef-5e63778a3955 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:44.101300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca9160-e752-472c-bcef-5e63778a3955 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:44.101582 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca9160-e752-472c-bcef-5e63778a3955 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:44.102316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca9160-e752-472c-bcef-5e63778a3955 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:44.102674 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca9160-e752-472c-bcef-5e63778a3955 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:44.104259 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca9160-e752-472c-bcef-5e63778a3955 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:44.116964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1127/2264] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:44.121340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372396fc-43de-460e-ae2f-80f40b376d75 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:44.131697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372396fc-43de-460e-ae2f-80f40b376d75 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:44.136705 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf3194-a1d7-428d-89c3-c2c40658a444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:44.137481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf3194-a1d7-428d-89c3-c2c40658a444 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:44.137738 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf3194-a1d7-428d-89c3-c2c40658a444 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:44.138107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf3194-a1d7-428d-89c3-c2c40658a444 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:44.138884 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf3194-a1d7-428d-89c3-c2c40658a444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:44.141472 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf3194-a1d7-428d-89c3-c2c40658a444 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:44.259757 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1138/2265] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:09:44 2022] POST /identity/v3/projects => generated 397 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:44.263756 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca9160-e752-472c-bcef-5e63778a3955 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:44.274716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca9160-e752-472c-bcef-5e63778a3955 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:44.302862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:44.336025 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0be90-e002-4c58-aaaa-df0dec8556ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:44.336710 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0be90-e002-4c58-aaaa-df0dec8556ce admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:44.336999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0be90-e002-4c58-aaaa-df0dec8556ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:44.337443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0be90-e002-4c58-aaaa-df0dec8556ce admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:44.337989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0be90-e002-4c58-aaaa-df0dec8556ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:44.339963 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0be90-e002-4c58-aaaa-df0dec8556ce admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:44.447580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1128/2266] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:09:44 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:44.449054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf3194-a1d7-428d-89c3-c2c40658a444 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:44.460709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf3194-a1d7-428d-89c3-c2c40658a444 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:44.465494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003e748-9df1-4a8b-bdc1-e875380c75eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:44.466216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e748-9df1-4a8b-bdc1-e875380c75eb admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:44.466579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e748-9df1-4a8b-bdc1-e875380c75eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:44.466944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e748-9df1-4a8b-bdc1-e875380c75eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:44.468214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003e748-9df1-4a8b-bdc1-e875380c75eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:44.470312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003e748-9df1-4a8b-bdc1-e875380c75eb admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:44.486358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1139/2267] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:09:44.490716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0be90-e002-4c58-aaaa-df0dec8556ce admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:44.501343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0be90-e002-4c58-aaaa-df0dec8556ce admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:44.506291 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3784365-a6c9-4a51-9023-d7775e507e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:44.506984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3784365-a6c9-4a51-9023-d7775e507e8e admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:44.507276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3784365-a6c9-4a51-9023-d7775e507e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:44.507558 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3784365-a6c9-4a51-9023-d7775e507e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:44.508287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3784365-a6c9-4a51-9023-d7775e507e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:44.509963 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3784365-a6c9-4a51-9023-d7775e507e8e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:44.521760 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1129/2268] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:09:44.529601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003e748-9df1-4a8b-bdc1-e875380c75eb admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:44.542676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003e748-9df1-4a8b-bdc1-e875380c75eb admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:44.548614 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcfa78-da9c-47ac-b81b-2aea619c2d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:44.549512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcfa78-da9c-47ac-b81b-2aea619c2d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:44.549797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcfa78-da9c-47ac-b81b-2aea619c2d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:44.550075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcfa78-da9c-47ac-b81b-2aea619c2d97 admin admin] PATH_INFO: `/v3/projects/9712f33d6d094716b83615402dce651a/users/af0ce0fde8fe4dd08cb687f9ba5e9300/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:44.569319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dcfa78-da9c-47ac-b81b-2aea619c2d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9712f33d6d094716b83615402dce651a, user_id=af0ce0fde8fe4dd08cb687f9ba5e9300, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:44.572025 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dcfa78-da9c-47ac-b81b-2aea619c2d97 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:44.651440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1140/2269] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:09:44 2022] PUT /identity/v3/projects/9712f33d6d094716b83615402dce651a/users/af0ce0fde8fe4dd08cb687f9ba5e9300/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:44.663382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e7c2b-c3e8-480e-914b-b9815ba81d69 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:44.663700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e7c2b-c3e8-480e-914b-b9815ba81d69 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:44.663975 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e7c2b-c3e8-480e-914b-b9815ba81d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:44.692926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b78e7c2b-c3e8-480e-914b-b9815ba81d69 None None] MFA Rules not processed for user `af0ce0fde8fe4dd08cb687f9ba5e9300`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:44.694165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b78e7c2b-c3e8-480e-914b-b9815ba81d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:44.761364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1130/2270] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:09:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:09:44.773869 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcfa78-da9c-47ac-b81b-2aea619c2d97 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:44.775716 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a0dcfa78-da9c-47ac-b81b-2aea619c2d97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:44.839241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcfa78-da9c-47ac-b81b-2aea619c2d97 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:44.843935 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edac516-cb77-4b92-a08b-88f011748f08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:44.844710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edac516-cb77-4b92-a08b-88f011748f08 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:44.845000 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edac516-cb77-4b92-a08b-88f011748f08 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:44.845286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edac516-cb77-4b92-a08b-88f011748f08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:44.855703 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edac516-cb77-4b92-a08b-88f011748f08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:44.857154 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edac516-cb77-4b92-a08b-88f011748f08 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:44.867196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1141/2271] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:09:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:09:54.758357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:54.862579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:07.185307 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e7c2b-c3e8-480e-914b-b9815ba81d69 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:07.186901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b78e7c2b-c3e8-480e-914b-b9815ba81d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:07.306804 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e7c2b-c3e8-480e-914b-b9815ba81d69 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:07.310466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8196e-36eb-4449-995e-4c0298b8a238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:07.311115 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8196e-36eb-4449-995e-4c0298b8a238 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:07.311398 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8196e-36eb-4449-995e-4c0298b8a238 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:07.311675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8196e-36eb-4449-995e-4c0298b8a238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:07.313558 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-32f8196e-36eb-4449-995e-4c0298b8a238 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:07.323938 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8196e-36eb-4449-995e-4c0298b8a238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:07.325364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8196e-36eb-4449-995e-4c0298b8a238 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:07.333298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1131/2272] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:10:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:10:12.265358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edac516-cb77-4b92-a08b-88f011748f08 service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:12.297978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edac516-cb77-4b92-a08b-88f011748f08 service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:12.303519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e6961-792c-439d-9631-e13017ceb13a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:12.304285 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e6961-792c-439d-9631-e13017ceb13a service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:12.304620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e6961-792c-439d-9631-e13017ceb13a service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:12.304955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e6961-792c-439d-9631-e13017ceb13a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:12.307043 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f92e6961-792c-439d-9631-e13017ceb13a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:12.421664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e6961-792c-439d-9631-e13017ceb13a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:12.423169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e6961-792c-439d-9631-e13017ceb13a service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:12.435967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1142/2273] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:10:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:10:13.010252 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8196e-36eb-4449-995e-4c0298b8a238 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:13.102535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8196e-36eb-4449-995e-4c0298b8a238 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:13.107914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:13.108796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:13.109089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:13.109401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] PATH_INFO: `/v3/users/af0ce0fde8fe4dd08cb687f9ba5e9300` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:13.110870 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af0ce0fde8fe4dd08cb687f9ba5e9300)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:13.113451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:13.436124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:13.468802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af0ce0fde8fe4dd08cb687f9ba5e9300'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:13.468802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af0ce0fde8fe4dd08cb687f9ba5e9300'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:13.468802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af0ce0fde8fe4dd08cb687f9ba5e9300'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:13.625331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af0ce0fde8fe4dd08cb687f9ba5e9300'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:13.639429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af0ce0fde8fe4dd08cb687f9ba5e9300'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:13.652977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1132/2274] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:10:13 2022] DELETE /identity/v3/users/af0ce0fde8fe4dd08cb687f9ba5e9300 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:10:13.657899 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23790c-6797-42be-a2ad-fec7f360421c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:13.658397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23790c-6797-42be-a2ad-fec7f360421c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:13.658541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23790c-6797-42be-a2ad-fec7f360421c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:13.701440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ea23790c-6797-42be-a2ad-fec7f360421c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:13.702630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ea23790c-6797-42be-a2ad-fec7f360421c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:13.763284 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1143/2275] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:10:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:10:13.774526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:13.776075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:13.837350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd740e6d-b445-498a-9101-78281e27f278 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:13.841510 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5e108-e44b-4e4a-88e0-41e27ff81f5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:13.842197 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5e108-e44b-4e4a-88e0-41e27ff81f5d service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:13.842524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5e108-e44b-4e4a-88e0-41e27ff81f5d service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:13.842799 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5e108-e44b-4e4a-88e0-41e27ff81f5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:13.854357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5e108-e44b-4e4a-88e0-41e27ff81f5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:13.855845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5e108-e44b-4e4a-88e0-41e27ff81f5d service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:13.866900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1133/2276] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:10:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:10:14.520156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23790c-6797-42be-a2ad-fec7f360421c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:14.521760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ea23790c-6797-42be-a2ad-fec7f360421c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:14.534263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23790c-6797-42be-a2ad-fec7f360421c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:14.540049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:14.541085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:14.541493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:14.541857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] PATH_INFO: `/v3/projects/9712f33d6d094716b83615402dce651a` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:14.543325 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9712f33d6d094716b83615402dce651a)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:14.546487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:14.575207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:14.800749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9712f33d6d094716b83615402dce651a', 'request_id': 'req-76676659-977b-49fe-8070-f47223087c18'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:14.807601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9712f33d6d094716b83615402dce651a', 'request_id': 'req-76676659-977b-49fe-8070-f47223087c18'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:14.813065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9712f33d6d094716b83615402dce651a', 'request_id': 'req-76676659-977b-49fe-8070-f47223087c18'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:15.110349 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] The token cache is being invalidate because project 9712f33d6d094716b83615402dce651a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:10:15.110762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:15.111419 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:15.113012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1144/2277] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:10:14 2022] DELETE /identity/v3/projects/9712f33d6d094716b83615402dce651a => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:10:23.861618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:DELETE=5,keystone:SELECT=40,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:25.107115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:26.367307 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5e108-e44b-4e4a-88e0-41e27ff81f5d service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:26.376648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5e108-e44b-4e4a-88e0-41e27ff81f5d service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:26.429285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787786fe-31a4-462b-9019-b71f196ebd48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:26.430005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787786fe-31a4-462b-9019-b71f196ebd48 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:26.430276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787786fe-31a4-462b-9019-b71f196ebd48 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:26.430602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787786fe-31a4-462b-9019-b71f196ebd48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:26.432617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-787786fe-31a4-462b-9019-b71f196ebd48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:26.509390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787786fe-31a4-462b-9019-b71f196ebd48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:26.510881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787786fe-31a4-462b-9019-b71f196ebd48 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:26.522847 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1134/2278] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:10:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:10:28.720925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:28.722471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:28.739338 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76676659-977b-49fe-8070-f47223087c18 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:28.745000 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb2617-2183-47dd-9463-47c24c244b95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:28.745655 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb2617-2183-47dd-9463-47c24c244b95 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:28.745948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb2617-2183-47dd-9463-47c24c244b95 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:28.747550 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb2617-2183-47dd-9463-47c24c244b95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:28.748148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fafb2617-2183-47dd-9463-47c24c244b95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:28.761128 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb2617-2183-47dd-9463-47c24c244b95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:28.762763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb2617-2183-47dd-9463-47c24c244b95 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:28.781855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1145/2279] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:10:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:10:36.517194 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=54 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:38.769078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=6 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:45.641801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787786fe-31a4-462b-9019-b71f196ebd48 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:45.643663 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-787786fe-31a4-462b-9019-b71f196ebd48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:45.727976 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787786fe-31a4-462b-9019-b71f196ebd48 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:45.733564 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:45.736784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:45.736784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:45.736784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] PATH_INFO: `/v3/users/6612b84e3ddb418d8f8d4c4ec7c10a8f` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:45.736784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6612b84e3ddb418d8f8d4c4ec7c10a8f)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:45.738259 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:46.003143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6612b84e3ddb418d8f8d4c4ec7c10a8f'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:46.003821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6612b84e3ddb418d8f8d4c4ec7c10a8f'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:46.004444 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6612b84e3ddb418d8f8d4c4ec7c10a8f'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:46.084925 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6612b84e3ddb418d8f8d4c4ec7c10a8f'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:46.100730 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6612b84e3ddb418d8f8d4c4ec7c10a8f'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:46.114598 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1135/2280] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:10:45 2022] DELETE /identity/v3/users/6612b84e3ddb418d8f8d4c4ec7c10a8f => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:10:46.119596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13c008-0e9f-4fad-8190-894e6e623bc1 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:46.120047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13c008-0e9f-4fad-8190-894e6e623bc1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:46.120047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13c008-0e9f-4fad-8190-894e6e623bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:46.143651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-5a13c008-0e9f-4fad-8190-894e6e623bc1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:46.144859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5a13c008-0e9f-4fad-8190-894e6e623bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:46.212943 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1146/2281] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:10:46.225537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:46.228060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:46.318860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36ae30-af10-47df-b745-fbd438ead14b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:46.323146 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd548a7-2915-4b9e-9b27-6f7a2884259a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:46.329488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd548a7-2915-4b9e-9b27-6f7a2884259a service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:46.329488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd548a7-2915-4b9e-9b27-6f7a2884259a service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:46.329488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd548a7-2915-4b9e-9b27-6f7a2884259a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:46.335771 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd548a7-2915-4b9e-9b27-6f7a2884259a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:46.337288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd548a7-2915-4b9e-9b27-6f7a2884259a service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:46.349000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1136/2282] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:10:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:10:46.910174 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13c008-0e9f-4fad-8190-894e6e623bc1 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:46.911839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5a13c008-0e9f-4fad-8190-894e6e623bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:46.926293 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13c008-0e9f-4fad-8190-894e6e623bc1 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:46.932608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:46.938001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:46.938001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:46.938001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c admin admin] PATH_INFO: `/v3/projects/4e489f01d3eb4d57847fd4153d42f620` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:46.938001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e489f01d3eb4d57847fd4153d42f620)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:46.938001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:47.093486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e489f01d3eb4d57847fd4153d42f620', 'request_id': 'req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:47.100088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e489f01d3eb4d57847fd4153d42f620', 'request_id': 'req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:47.106253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e489f01d3eb4d57847fd4153d42f620', 'request_id': 'req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:47.206053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c admin admin] The token cache is being invalidate because project 4e489f01d3eb4d57847fd4153d42f620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:10:47.206746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:47.207064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-30368a9e-c471-46d9-bd6d-ed0132bb3f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:47.208937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1147/2283] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:10:46 2022] DELETE /identity/v3/projects/4e489f01d3eb4d57847fd4153d42f620 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:10:47.215282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3fc5f-5e89-4c63-a274-118a5593655f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:47.216064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3fc5f-5e89-4c63-a274-118a5593655f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:47.216064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3fc5f-5e89-4c63-a274-118a5593655f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:47.263704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:47.280450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-7cb3fc5f-5e89-4c63-a274-118a5593655f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:47.281579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-7cb3fc5f-5e89-4c63-a274-118a5593655f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:47.355754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1137/2284] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:10:47.366447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a96fae-ec78-425c-900f-1ab27484daf6 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:47.368369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a96fae-ec78-425c-900f-1ab27484daf6 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:47.368369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a96fae-ec78-425c-900f-1ab27484daf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:47.394679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-94a96fae-ec78-425c-900f-1ab27484daf6 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:47.396528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-94a96fae-ec78-425c-900f-1ab27484daf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:47.407622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1148/2285] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:10:47.413800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3fc5f-5e89-4c63-a274-118a5593655f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:47.433959 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3fc5f-5e89-4c63-a274-118a5593655f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:47.441239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9a67f-61d9-4ba8-83ab-d5f4d4992117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:47.442074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9a67f-61d9-4ba8-83ab-d5f4d4992117 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:47.442407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9a67f-61d9-4ba8-83ab-d5f4d4992117 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:47.442699 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9a67f-61d9-4ba8-83ab-d5f4d4992117 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:47.443482 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9a67f-61d9-4ba8-83ab-d5f4d4992117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:47.443827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9a67f-61d9-4ba8-83ab-d5f4d4992117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:47.445459 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9a67f-61d9-4ba8-83ab-d5f4d4992117 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:47.460101 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1138/2286] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:47.465136 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a96fae-ec78-425c-900f-1ab27484daf6 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:47.475085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a96fae-ec78-425c-900f-1ab27484daf6 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:47.480592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9117e0-7837-4125-a569-4002f0fbbaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:47.481271 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9117e0-7837-4125-a569-4002f0fbbaf3 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:47.481553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9117e0-7837-4125-a569-4002f0fbbaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:47.481829 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9117e0-7837-4125-a569-4002f0fbbaf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:47.482572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9117e0-7837-4125-a569-4002f0fbbaf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:47.484484 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9117e0-7837-4125-a569-4002f0fbbaf3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:47.641700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1149/2287] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:10:47 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:10:47.646054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9a67f-61d9-4ba8-83ab-d5f4d4992117 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:47.656358 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9a67f-61d9-4ba8-83ab-d5f4d4992117 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:47.759219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502fa45-1d06-4783-a201-53c4a607a4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:47.759913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502fa45-1d06-4783-a201-53c4a607a4fb admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:47.760199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502fa45-1d06-4783-a201-53c4a607a4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:47.760573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502fa45-1d06-4783-a201-53c4a607a4fb admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:47.761579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502fa45-1d06-4783-a201-53c4a607a4fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:47.764626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502fa45-1d06-4783-a201-53c4a607a4fb admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:47.895855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1139/2288] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:10:47 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:10:47.902411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9117e0-7837-4125-a569-4002f0fbbaf3 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:47.914788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9117e0-7837-4125-a569-4002f0fbbaf3 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:47.919933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b0c1c-46b8-45b7-8200-b4854f0aa08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:47.920647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b0c1c-46b8-45b7-8200-b4854f0aa08d admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:47.920936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b0c1c-46b8-45b7-8200-b4854f0aa08d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:47.921221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b0c1c-46b8-45b7-8200-b4854f0aa08d admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:47.922011 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b0c1c-46b8-45b7-8200-b4854f0aa08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:47.923686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b0c1c-46b8-45b7-8200-b4854f0aa08d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:47.935301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1150/2289] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:10:47.939827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502fa45-1d06-4783-a201-53c4a607a4fb admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:47.953103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502fa45-1d06-4783-a201-53c4a607a4fb admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:47.958269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f6ae4-9559-4371-a45e-da07b7e2fac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:47.959007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f6ae4-9559-4371-a45e-da07b7e2fac7 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:47.959297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f6ae4-9559-4371-a45e-da07b7e2fac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:47.959578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f6ae4-9559-4371-a45e-da07b7e2fac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:47.960320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f6ae4-9559-4371-a45e-da07b7e2fac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:47.962761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f6ae4-9559-4371-a45e-da07b7e2fac7 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:47.974597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1140/2290] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:10:47.980899 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b0c1c-46b8-45b7-8200-b4854f0aa08d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:47.992207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b0c1c-46b8-45b7-8200-b4854f0aa08d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:47.997589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5d1c3-5461-46ff-9f0d-924a6df206f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:47.998552 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5d1c3-5461-46ff-9f0d-924a6df206f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:47.998841 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5d1c3-5461-46ff-9f0d-924a6df206f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:47.999119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5d1c3-5461-46ff-9f0d-924a6df206f4 admin admin] PATH_INFO: `/v3/projects/8716243e827148c5a9275ac00f50d90a/users/f417ce675fbe41af86baea3a086582f8/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:48.015606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5d1c3-5461-46ff-9f0d-924a6df206f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8716243e827148c5a9275ac00f50d90a, user_id=f417ce675fbe41af86baea3a086582f8, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:48.018263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5d1c3-5461-46ff-9f0d-924a6df206f4 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:48.132276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1151/2291] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:10:47 2022] PUT /identity/v3/projects/8716243e827148c5a9275ac00f50d90a/users/f417ce675fbe41af86baea3a086582f8/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:10:48.202970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c999-c00e-438e-b3d0-2090427c2325 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:48.203201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c999-c00e-438e-b3d0-2090427c2325 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:48.203453 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c999-c00e-438e-b3d0-2090427c2325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:48.240178 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c814c999-c00e-438e-b3d0-2090427c2325 None None] MFA Rules not processed for user `f417ce675fbe41af86baea3a086582f8`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:48.241725 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c814c999-c00e-438e-b3d0-2090427c2325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:48.307955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:48.320020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1141/2292] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:10:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:10:48.333431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5d1c3-5461-46ff-9f0d-924a6df206f4 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:48.335592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b6e5d1c3-5461-46ff-9f0d-924a6df206f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:48.352225 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:48.419311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5d1c3-5461-46ff-9f0d-924a6df206f4 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:48.423324 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9d93a-d85f-4658-90fc-9a0c37e6b156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:48.423978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9d93a-d85f-4658-90fc-9a0c37e6b156 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:48.424256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9d93a-d85f-4658-90fc-9a0c37e6b156 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:48.424536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9d93a-d85f-4658-90fc-9a0c37e6b156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:48.434791 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9d93a-d85f-4658-90fc-9a0c37e6b156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:48.436284 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9d93a-d85f-4658-90fc-9a0c37e6b156 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:48.447207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1152/2293] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:10:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:10:48.473939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c999-c00e-438e-b3d0-2090427c2325 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:48.543977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c999-c00e-438e-b3d0-2090427c2325 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:48.548871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:48.549760 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:48.550043 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:48.550343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] PATH_INFO: `/v3/users/f417ce675fbe41af86baea3a086582f8` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:48.551670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f417ce675fbe41af86baea3a086582f8)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:48.553594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:48.803290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f417ce675fbe41af86baea3a086582f8'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:48.804121 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f417ce675fbe41af86baea3a086582f8'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:48.804814 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f417ce675fbe41af86baea3a086582f8'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:48.886817 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f417ce675fbe41af86baea3a086582f8'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:48.901403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f417ce675fbe41af86baea3a086582f8'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:48.914854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1142/2294] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:10:48 2022] DELETE /identity/v3/users/f417ce675fbe41af86baea3a086582f8 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:10:48.919416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d7761-85ba-4d5d-a87e-ed2fcafb7858 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:48.919687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d7761-85ba-4d5d-a87e-ed2fcafb7858 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:48.919973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d7761-85ba-4d5d-a87e-ed2fcafb7858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:48.939710 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-f74d7761-85ba-4d5d-a87e-ed2fcafb7858 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:48.940922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f74d7761-85ba-4d5d-a87e-ed2fcafb7858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:49.002980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1153/2295] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:10:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:10:49.013947 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:49.015465 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:49.080327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf3ba4-c0b5-4a16-a4ac-01dec4d0be29 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:49.084095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecaae03-2841-4d53-8af1-579c51554365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:49.084767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaae03-2841-4d53-8af1-579c51554365 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:49.085053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaae03-2841-4d53-8af1-579c51554365 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:49.085336 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaae03-2841-4d53-8af1-579c51554365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:49.098062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecaae03-2841-4d53-8af1-579c51554365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:49.099573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecaae03-2841-4d53-8af1-579c51554365 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:49.113815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1143/2296] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:10:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:10:49.650736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d7761-85ba-4d5d-a87e-ed2fcafb7858 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:49.665753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f74d7761-85ba-4d5d-a87e-ed2fcafb7858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:49.673782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d7761-85ba-4d5d-a87e-ed2fcafb7858 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:49.679293 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:49.680557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:49.681563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:49.681995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] PATH_INFO: `/v3/projects/8716243e827148c5a9275ac00f50d90a` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:49.684044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8716243e827148c5a9275ac00f50d90a)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:49.686802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:49.851290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8716243e827148c5a9275ac00f50d90a', 'request_id': 'req-e066cbd3-b988-463e-bb3b-a09bc99090e1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:49.860101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8716243e827148c5a9275ac00f50d90a', 'request_id': 'req-e066cbd3-b988-463e-bb3b-a09bc99090e1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:49.867099 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8716243e827148c5a9275ac00f50d90a', 'request_id': 'req-e066cbd3-b988-463e-bb3b-a09bc99090e1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:49.955982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] The token cache is being invalidate because project 8716243e827148c5a9275ac00f50d90a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:10:49.956401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:49.957249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:49.959460 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1154/2297] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:10:49 2022] DELETE /identity/v3/projects/8716243e827148c5a9275ac00f50d90a => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:10:51.236423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df614cf8-2cb6-4535-b843-6c9fbbcec380 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:51.237243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df614cf8-2cb6-4535-b843-6c9fbbcec380 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:51.237243 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df614cf8-2cb6-4535-b843-6c9fbbcec380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:51.259398 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-df614cf8-2cb6-4535-b843-6c9fbbcec380 None None] MFA Rules not processed for user `5321524c39c14135b5cc77e21598cbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:51.261043 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-df614cf8-2cb6-4535-b843-6c9fbbcec380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:51.273042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:51.327468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1144/2298] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:10:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:10:51.584257 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:51.585700 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:51.620584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:51.649757 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066cbd3-b988-463e-bb3b-a09bc99090e1 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:51.653301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23d387-0b5a-4b2a-931c-481ee0247c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:51.653978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23d387-0b5a-4b2a-931c-481ee0247c38 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:51.654269 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23d387-0b5a-4b2a-931c-481ee0247c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:51.654579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23d387-0b5a-4b2a-931c-481ee0247c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:51.656395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-2a23d387-0b5a-4b2a-931c-481ee0247c38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:51.667056 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23d387-0b5a-4b2a-931c-481ee0247c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:51.668934 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23d387-0b5a-4b2a-931c-481ee0247c38 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:51.676850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1155/2299] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:10:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:10:53.237083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df614cf8-2cb6-4535-b843-6c9fbbcec380 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:53.240458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-df614cf8-2cb6-4535-b843-6c9fbbcec380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:53.353212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df614cf8-2cb6-4535-b843-6c9fbbcec380 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:53.358412 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ab63d-26dd-4ae8-bdb0-34e4d93c5d9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:53.359081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ab63d-26dd-4ae8-bdb0-34e4d93c5d9a service placement] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:53.359364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ab63d-26dd-4ae8-bdb0-34e4d93c5d9a service placement] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:53.359632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ab63d-26dd-4ae8-bdb0-34e4d93c5d9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:53.361689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-dc2ab63d-26dd-4ae8-bdb0-34e4d93c5d9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:53.379335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ab63d-26dd-4ae8-bdb0-34e4d93c5d9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:53.381011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ab63d-26dd-4ae8-bdb0-34e4d93c5d9a service placement] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:53.391816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1145/2300] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:10:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:10:54.852647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23d387-0b5a-4b2a-931c-481ee0247c38 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:54.854600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-2a23d387-0b5a-4b2a-931c-481ee0247c38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:54.939336 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23d387-0b5a-4b2a-931c-481ee0247c38 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:54.944450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:54.945490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:54.945832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:54.946720 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] PATH_INFO: `/v3/users/96ff65becd4b47c69bb582b058e160ca` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:54.948363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] RBAC: Authorizing `identity:delete_user(user_id=96ff65becd4b47c69bb582b058e160ca)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:54.950612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:55.171279 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96ff65becd4b47c69bb582b058e160ca'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:55.173237 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96ff65becd4b47c69bb582b058e160ca'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:55.179190 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96ff65becd4b47c69bb582b058e160ca'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:55.193076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96ff65becd4b47c69bb582b058e160ca'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:55.276031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96ff65becd4b47c69bb582b058e160ca'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:55.288515 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1156/2301] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:10:54 2022] DELETE /identity/v3/users/96ff65becd4b47c69bb582b058e160ca => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:10:55.293881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939d072-df0d-4259-82fe-21420bc74d21 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:55.294526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939d072-df0d-4259-82fe-21420bc74d21 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:55.294526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939d072-df0d-4259-82fe-21420bc74d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:55.315288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b939d072-df0d-4259-82fe-21420bc74d21 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:55.316558 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b939d072-df0d-4259-82fe-21420bc74d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:55.393815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1146/2302] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:10:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:10:55.411887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:55.414183 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:55.509140 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:55.513905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a01b3-4a89-4827-9a00-de91d24fe37c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:55.524841 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e20e76-ef92-47e7-9071-5d040a6f8dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:55.533466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e20e76-ef92-47e7-9071-5d040a6f8dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:55.533466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e20e76-ef92-47e7-9071-5d040a6f8dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:55.533466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e20e76-ef92-47e7-9071-5d040a6f8dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:55.543801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e20e76-ef92-47e7-9071-5d040a6f8dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:55.545441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e20e76-ef92-47e7-9071-5d040a6f8dd1 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:55.556830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1157/2303] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:10:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:10:56.109541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939d072-df0d-4259-82fe-21420bc74d21 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:56.111009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b939d072-df0d-4259-82fe-21420bc74d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:56.126226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939d072-df0d-4259-82fe-21420bc74d21 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:56.135998 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:56.138890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:56.138890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:56.138890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] PATH_INFO: `/v3/projects/bf9af35f313c48efba86e1a3ae0bc23c` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:56.140342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf9af35f313c48efba86e1a3ae0bc23c)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:56.141327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:56.556154 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:56.558998 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf9af35f313c48efba86e1a3ae0bc23c', 'request_id': 'req-7335f003-6017-4787-bf53-639821380ac4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:56.669629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf9af35f313c48efba86e1a3ae0bc23c', 'request_id': 'req-7335f003-6017-4787-bf53-639821380ac4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:56.677215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf9af35f313c48efba86e1a3ae0bc23c', 'request_id': 'req-7335f003-6017-4787-bf53-639821380ac4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:56.690289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] The token cache is being invalidate because project bf9af35f313c48efba86e1a3ae0bc23c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:10:56.693459 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:56.693459 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:10:56.693459 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1147/2304] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:10:56 2022] DELETE /identity/v3/projects/bf9af35f313c48efba86e1a3ae0bc23c => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:11:05.836293 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:06.713529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:09.046600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e20e76-ef92-47e7-9071-5d040a6f8dd1 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:09.048336 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d7e20e76-ef92-47e7-9071-5d040a6f8dd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:11:09.128573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e20e76-ef92-47e7-9071-5d040a6f8dd1 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:09.207682 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d687718-7494-483b-b6fa-565913f075b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:09.208351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d687718-7494-483b-b6fa-565913f075b5 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:09.208648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d687718-7494-483b-b6fa-565913f075b5 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:09.208922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d687718-7494-483b-b6fa-565913f075b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:09.210818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7d687718-7494-483b-b6fa-565913f075b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:11:09.223779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d687718-7494-483b-b6fa-565913f075b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:09.225381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d687718-7494-483b-b6fa-565913f075b5 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:09.237191 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1158/2305] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:11:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:11:19.232642 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=62 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:31.269759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:31.271357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:11:31.289686 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335f003-6017-4787-bf53-639821380ac4 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:31.296036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d108b9-c31d-4bdb-991d-a9f2a54b3947 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:31.296690 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d108b9-c31d-4bdb-991d-a9f2a54b3947 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:31.296978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d108b9-c31d-4bdb-991d-a9f2a54b3947 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:31.297258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d108b9-c31d-4bdb-991d-a9f2a54b3947 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:31.299013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a4d108b9-c31d-4bdb-991d-a9f2a54b3947 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:11:31.362438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d108b9-c31d-4bdb-991d-a9f2a54b3947 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:31.364099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d108b9-c31d-4bdb-991d-a9f2a54b3947 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:31.374109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1148/2306] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:11:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:11:41.369626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=28 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:41.975683 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d687718-7494-483b-b6fa-565913f075b5 service nova] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:41.977192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7d687718-7494-483b-b6fa-565913f075b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:11:42.055592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d687718-7494-483b-b6fa-565913f075b5 service nova] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:42.060440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac17ae-4944-4ba3-8c6f-3fa1a9daac5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:42.061118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac17ae-4944-4ba3-8c6f-3fa1a9daac5c service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:42.061407 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac17ae-4944-4ba3-8c6f-3fa1a9daac5c service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:42.061687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac17ae-4944-4ba3-8c6f-3fa1a9daac5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:42.064148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-beac17ae-4944-4ba3-8c6f-3fa1a9daac5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:11:42.140545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac17ae-4944-4ba3-8c6f-3fa1a9daac5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:42.142044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac17ae-4944-4ba3-8c6f-3fa1a9daac5c service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:42.152965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1159/2307] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:11:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:11:52.147704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=48 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:09.880124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d108b9-c31d-4bdb-991d-a9f2a54b3947 service nova] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:09.881765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a4d108b9-c31d-4bdb-991d-a9f2a54b3947 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:09.927512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d108b9-c31d-4bdb-991d-a9f2a54b3947 service nova] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:09.933124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b46869-0030-42b6-b2e8-38cc3d13d5c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:09.933792 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b46869-0030-42b6-b2e8-38cc3d13d5c1 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:09.934081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b46869-0030-42b6-b2e8-38cc3d13d5c1 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:09.934384 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b46869-0030-42b6-b2e8-38cc3d13d5c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:09.936343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-64b46869-0030-42b6-b2e8-38cc3d13d5c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:09.997114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b46869-0030-42b6-b2e8-38cc3d13d5c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:09.999493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b46869-0030-42b6-b2e8-38cc3d13d5c1 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:10.009171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1149/2308] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:12:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:12:14.609206 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7a5a3-2960-4538-b4bc-9500113ec8a4 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:14.609969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7a5a3-2960-4538-b4bc-9500113ec8a4 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:14.609969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7a5a3-2960-4538-b4bc-9500113ec8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:14.643551 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-8ae7a5a3-2960-4538-b4bc-9500113ec8a4 None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:14.644824 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8ae7a5a3-2960-4538-b4bc-9500113ec8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:14.712186 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1160/2309] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:12:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:12:14.894947 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b46869-0030-42b6-b2e8-38cc3d13d5c1 service nova] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:14.896512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-64b46869-0030-42b6-b2e8-38cc3d13d5c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:14.974504 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b46869-0030-42b6-b2e8-38cc3d13d5c1 service nova] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:14.979624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092834b-4966-4974-be4d-b2417c9abb8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:14.980329 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092834b-4966-4974-be4d-b2417c9abb8e service swift] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:14.980618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092834b-4966-4974-be4d-b2417c9abb8e service swift] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:14.981020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092834b-4966-4974-be4d-b2417c9abb8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:14.990061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092834b-4966-4974-be4d-b2417c9abb8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:14.991813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092834b-4966-4974-be4d-b2417c9abb8e service swift] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:15.000142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1150/2310] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:12:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:12:23.423132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7a5a3-2960-4538-b4bc-9500113ec8a4 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:23.425426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8ae7a5a3-2960-4538-b4bc-9500113ec8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:23.509631 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7a5a3-2960-4538-b4bc-9500113ec8a4 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:23.513559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96824932-f7d3-4aae-a462-7119c7f93e69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:23.514229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96824932-f7d3-4aae-a462-7119c7f93e69 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:23.514685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96824932-f7d3-4aae-a462-7119c7f93e69 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:23.515028 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96824932-f7d3-4aae-a462-7119c7f93e69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:23.516883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-96824932-f7d3-4aae-a462-7119c7f93e69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:23.636980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96824932-f7d3-4aae-a462-7119c7f93e69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:23.638643 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96824932-f7d3-4aae-a462-7119c7f93e69 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:23.655141 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1161/2311] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:12:24.352094 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092834b-4966-4974-be4d-b2417c9abb8e service swift] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:24.354289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8092834b-4966-4974-be4d-b2417c9abb8e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:24.487175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:24.537304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092834b-4966-4974-be4d-b2417c9abb8e service swift] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:24.544823 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b423d2-b7fd-4676-9c03-944e650437c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:24.545622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b423d2-b7fd-4676-9c03-944e650437c2 service placement] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:24.546600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b423d2-b7fd-4676-9c03-944e650437c2 service placement] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:24.546989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b423d2-b7fd-4676-9c03-944e650437c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:24.556811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e2b423d2-b7fd-4676-9c03-944e650437c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:24.564403 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96824932-f7d3-4aae-a462-7119c7f93e69 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:24.567135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-96824932-f7d3-4aae-a462-7119c7f93e69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:24.593352 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96824932-f7d3-4aae-a462-7119c7f93e69 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:24.597628 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b423d2-b7fd-4676-9c03-944e650437c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:24.599474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b423d2-b7fd-4676-9c03-944e650437c2 service placement] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:24.601659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa920e-9690-455c-b7bb-a1d6dd289c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:24.603537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa920e-9690-455c-b7bb-a1d6dd289c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:24.603838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa920e-9690-455c-b7bb-a1d6dd289c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:24.604122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa920e-9690-455c-b7bb-a1d6dd289c68 admin admin] PATH_INFO: `/v3/users/5321524c39c14135b5cc77e21598cbe1` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:24.605575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa920e-9690-455c-b7bb-a1d6dd289c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5321524c39c14135b5cc77e21598cbe1)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:24.610531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa920e-9690-455c-b7bb-a1d6dd289c68 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:24.625892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1151/2312] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:12:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:12:24.838799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:24.846271 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ceaa920e-9690-455c-b7bb-a1d6dd289c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5321524c39c14135b5cc77e21598cbe1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:12:24.847282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ceaa920e-9690-455c-b7bb-a1d6dd289c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5321524c39c14135b5cc77e21598cbe1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:12:24.857499 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ceaa920e-9690-455c-b7bb-a1d6dd289c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5321524c39c14135b5cc77e21598cbe1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:12:24.871953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ceaa920e-9690-455c-b7bb-a1d6dd289c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5321524c39c14135b5cc77e21598cbe1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:12:24.953487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ceaa920e-9690-455c-b7bb-a1d6dd289c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5321524c39c14135b5cc77e21598cbe1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:12:24.961791 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1162/2313] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:12:24 2022] DELETE /identity/v3/users/5321524c39c14135b5cc77e21598cbe1 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:12:25.259251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b423d2-b7fd-4676-9c03-944e650437c2 service placement] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:25.273962 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b423d2-b7fd-4676-9c03-944e650437c2 service placement] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:25.354188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927adf71-2977-41b8-bbff-3e03be548fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:25.355240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927adf71-2977-41b8-bbff-3e03be548fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:25.355546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927adf71-2977-41b8-bbff-3e03be548fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:25.355830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927adf71-2977-41b8-bbff-3e03be548fb8 admin admin] PATH_INFO: `/v3/projects/387ac04100224395a581aa382deb19b0` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:25.357053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927adf71-2977-41b8-bbff-3e03be548fb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=387ac04100224395a581aa382deb19b0)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:25.360415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927adf71-2977-41b8-bbff-3e03be548fb8 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:25.735055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-927adf71-2977-41b8-bbff-3e03be548fb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387ac04100224395a581aa382deb19b0', 'request_id': 'req-927adf71-2977-41b8-bbff-3e03be548fb8'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:12:25.861210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-927adf71-2977-41b8-bbff-3e03be548fb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387ac04100224395a581aa382deb19b0', 'request_id': 'req-927adf71-2977-41b8-bbff-3e03be548fb8'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:12:25.867417 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-927adf71-2977-41b8-bbff-3e03be548fb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387ac04100224395a581aa382deb19b0', 'request_id': 'req-927adf71-2977-41b8-bbff-3e03be548fb8'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:12:25.880857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-927adf71-2977-41b8-bbff-3e03be548fb8 admin admin] The token cache is being invalidate because project 387ac04100224395a581aa382deb19b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:12:25.881185 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-927adf71-2977-41b8-bbff-3e03be548fb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:12:25.882590 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-927adf71-2977-41b8-bbff-3e03be548fb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:12:25.884215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1152/2314] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:12:25 2022] DELETE /identity/v3/projects/387ac04100224395a581aa382deb19b0 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:12:25.890783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0bccb-d8a3-4b45-9964-4baa628c19eb None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:25.891313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0bccb-d8a3-4b45-9964-4baa628c19eb None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:25.891436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0bccb-d8a3-4b45-9964-4baa628c19eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:25.911034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-9ca0bccb-d8a3-4b45-9964-4baa628c19eb None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:25.912158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9ca0bccb-d8a3-4b45-9964-4baa628c19eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:25.976825 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1163/2315] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:12:25.985344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e402ff2-713c-4563-ad39-87b0889069cb None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:25.985627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e402ff2-713c-4563-ad39-87b0889069cb None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:25.985989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e402ff2-713c-4563-ad39-87b0889069cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:26.014102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-0e402ff2-713c-4563-ad39-87b0889069cb None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:26.016212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0e402ff2-713c-4563-ad39-87b0889069cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:26.024648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1153/2316] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:12:26.029612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0bccb-d8a3-4b45-9964-4baa628c19eb None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:26.039791 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0bccb-d8a3-4b45-9964-4baa628c19eb None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:26.044470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3612c-4332-4271-b5a5-559acfc801db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:26.045132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3612c-4332-4271-b5a5-559acfc801db admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:26.045416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3612c-4332-4271-b5a5-559acfc801db admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:26.045765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3612c-4332-4271-b5a5-559acfc801db admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:26.046705 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3612c-4332-4271-b5a5-559acfc801db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:26.046925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3612c-4332-4271-b5a5-559acfc801db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:26.048527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3612c-4332-4271-b5a5-559acfc801db admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:26.068037 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1164/2317] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:12:26.069283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e402ff2-713c-4563-ad39-87b0889069cb None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:26.080416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e402ff2-713c-4563-ad39-87b0889069cb None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:26.085331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d52f49-49ea-4850-9f45-24e3a6cca6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:26.086075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d52f49-49ea-4850-9f45-24e3a6cca6fb admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:26.086467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d52f49-49ea-4850-9f45-24e3a6cca6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:26.086822 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d52f49-49ea-4850-9f45-24e3a6cca6fb admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:26.087608 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d52f49-49ea-4850-9f45-24e3a6cca6fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:26.089828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d52f49-49ea-4850-9f45-24e3a6cca6fb admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:26.200475 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1154/2318] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:12:26 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:12:26.204800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3612c-4332-4271-b5a5-559acfc801db admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:26.215186 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3612c-4332-4271-b5a5-559acfc801db admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:26.287747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1918522-db49-4086-9fee-7c49f4da6284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:26.288428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1918522-db49-4086-9fee-7c49f4da6284 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:26.288716 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1918522-db49-4086-9fee-7c49f4da6284 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:26.289176 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1918522-db49-4086-9fee-7c49f4da6284 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:26.289927 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1918522-db49-4086-9fee-7c49f4da6284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:26.291941 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1918522-db49-4086-9fee-7c49f4da6284 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:26.385670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1165/2319] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:12:26 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:12:26.389489 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d52f49-49ea-4850-9f45-24e3a6cca6fb admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:26.402228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d52f49-49ea-4850-9f45-24e3a6cca6fb admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:26.414485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5877248-d528-47cb-89da-8e6a0abf0363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:26.414485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5877248-d528-47cb-89da-8e6a0abf0363 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:26.414950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5877248-d528-47cb-89da-8e6a0abf0363 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:26.414950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5877248-d528-47cb-89da-8e6a0abf0363 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:26.415603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5877248-d528-47cb-89da-8e6a0abf0363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:26.417319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5877248-d528-47cb-89da-8e6a0abf0363 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:26.432413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1155/2320] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:12:26.437516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1918522-db49-4086-9fee-7c49f4da6284 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:26.451740 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1918522-db49-4086-9fee-7c49f4da6284 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:26.456849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246a1a2-11a1-4724-a2a0-928b82d01f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:26.457697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246a1a2-11a1-4724-a2a0-928b82d01f00 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:26.458100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246a1a2-11a1-4724-a2a0-928b82d01f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:26.458546 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246a1a2-11a1-4724-a2a0-928b82d01f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:26.459535 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246a1a2-11a1-4724-a2a0-928b82d01f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:26.461231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246a1a2-11a1-4724-a2a0-928b82d01f00 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:26.473245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1166/2321] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:12:26.477362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5877248-d528-47cb-89da-8e6a0abf0363 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:26.489429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5877248-d528-47cb-89da-8e6a0abf0363 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:26.497254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63620e60-be22-423f-853d-6f61fb63c31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:26.498209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63620e60-be22-423f-853d-6f61fb63c31b admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:26.498537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63620e60-be22-423f-853d-6f61fb63c31b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:26.498914 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63620e60-be22-423f-853d-6f61fb63c31b admin admin] PATH_INFO: `/v3/projects/019f72079acf4a74bd1a0706041e6678/users/9631ec5df93c42df9d2f60bedfef6507/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:26.517063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63620e60-be22-423f-853d-6f61fb63c31b admin admin] RBAC: Authorizing `identity:create_grant(project_id=019f72079acf4a74bd1a0706041e6678, user_id=9631ec5df93c42df9d2f60bedfef6507, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:26.519969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63620e60-be22-423f-853d-6f61fb63c31b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:26.645565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:26.651858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1156/2322] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:12:26 2022] PUT /identity/v3/projects/019f72079acf4a74bd1a0706041e6678/users/9631ec5df93c42df9d2f60bedfef6507/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:12:26.657954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa197f-5799-4798-967e-851bea9ff5c7 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:26.658501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa197f-5799-4798-967e-851bea9ff5c7 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:26.658923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa197f-5799-4798-967e-851bea9ff5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:26.681819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-86fa197f-5799-4798-967e-851bea9ff5c7 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:26.683258 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-86fa197f-5799-4798-967e-851bea9ff5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:26.708573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:26.759379 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1167/2323] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:12:26.767600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63620e60-be22-423f-853d-6f61fb63c31b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:26.769398 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-63620e60-be22-423f-853d-6f61fb63c31b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:26.884578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63620e60-be22-423f-853d-6f61fb63c31b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:26.888773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554993b-4715-467f-88a6-496640286269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:26.890566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554993b-4715-467f-88a6-496640286269 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:26.892069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554993b-4715-467f-88a6-496640286269 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:26.892476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554993b-4715-467f-88a6-496640286269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:26.903311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554993b-4715-467f-88a6-496640286269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:26.904907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554993b-4715-467f-88a6-496640286269 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:26.917461 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1157/2324] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:12:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:12:30.785942 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa197f-5799-4798-967e-851bea9ff5c7 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:30.788596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-86fa197f-5799-4798-967e-851bea9ff5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:30.886799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa197f-5799-4798-967e-851bea9ff5c7 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:30.894920 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696283da-4546-4a00-b09a-a74caa35da01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:30.896527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696283da-4546-4a00-b09a-a74caa35da01 service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:30.896836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696283da-4546-4a00-b09a-a74caa35da01 service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:30.897125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696283da-4546-4a00-b09a-a74caa35da01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:30.900164 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-696283da-4546-4a00-b09a-a74caa35da01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:30.916211 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696283da-4546-4a00-b09a-a74caa35da01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:30.918361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696283da-4546-4a00-b09a-a74caa35da01 service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:30.945486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1168/2325] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:12:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:12:34.172307 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8be1cf-7f91-4f66-af6e-d74e56d0ac4d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.172307 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8be1cf-7f91-4f66-af6e-d74e56d0ac4d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.172307 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8be1cf-7f91-4f66-af6e-d74e56d0ac4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.245364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3c8be1cf-7f91-4f66-af6e-d74e56d0ac4d None None] MFA Rules not processed for user `9631ec5df93c42df9d2f60bedfef6507`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:34.247254 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3c8be1cf-7f91-4f66-af6e-d74e56d0ac4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:34.357541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1158/2326] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:12:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:12:34.362915 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696283da-4546-4a00-b09a-a74caa35da01 service placement] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:34.376436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696283da-4546-4a00-b09a-a74caa35da01 service placement] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:34.381641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4914530-c132-429a-a7a0-31bd193d3286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:34.382351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4914530-c132-429a-a7a0-31bd193d3286 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.388816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4914530-c132-429a-a7a0-31bd193d3286 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.388816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4914530-c132-429a-a7a0-31bd193d3286 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.388816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4914530-c132-429a-a7a0-31bd193d3286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:34.388816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4914530-c132-429a-a7a0-31bd193d3286 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:34.476806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1169/2327] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:12:34 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:12:34.480863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8be1cf-7f91-4f66-af6e-d74e56d0ac4d None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:34.493534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8be1cf-7f91-4f66-af6e-d74e56d0ac4d None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:34.573553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d99980-9131-409e-8a29-a2703c41b1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:34.574596 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d99980-9131-409e-8a29-a2703c41b1be admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.574884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d99980-9131-409e-8a29-a2703c41b1be admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.575165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d99980-9131-409e-8a29-a2703c41b1be admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.577704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d99980-9131-409e-8a29-a2703c41b1be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:34.579708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d99980-9131-409e-8a29-a2703c41b1be admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:34.641795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05730c-f4cc-4a7d-9cd0-75cee82ed1e1 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.641795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05730c-f4cc-4a7d-9cd0-75cee82ed1e1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.641795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05730c-f4cc-4a7d-9cd0-75cee82ed1e1 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.641795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1170/2328] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 19:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:12:34.644303 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdad0e-d940-4aa4-acfc-d79caddb7119 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.644667 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdad0e-d940-4aa4-acfc-d79caddb7119 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.645033 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdad0e-d940-4aa4-acfc-d79caddb7119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.671806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-5dcdad0e-d940-4aa4-acfc-d79caddb7119 None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:34.671806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5dcdad0e-d940-4aa4-acfc-d79caddb7119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:34.703225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1159/2329] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:12:34 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:12:34.706875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d99980-9131-409e-8a29-a2703c41b1be admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:34.721048 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d99980-9131-409e-8a29-a2703c41b1be admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:34.734938 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24105748-fbf4-4192-8040-8de28f230407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:34.736210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24105748-fbf4-4192-8040-8de28f230407 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.736504 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24105748-fbf4-4192-8040-8de28f230407 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.736791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24105748-fbf4-4192-8040-8de28f230407 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.737560 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24105748-fbf4-4192-8040-8de28f230407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:34.739269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24105748-fbf4-4192-8040-8de28f230407 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:34.750825 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1160/2330] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:12:34.754805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24105748-fbf4-4192-8040-8de28f230407 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:34.763220 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1171/2331] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 19:12:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:12:34.765611 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24105748-fbf4-4192-8040-8de28f230407 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:34.771989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1f0c6-1c8c-4c8c-96ab-7c5472a3a16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:34.773094 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1f0c6-1c8c-4c8c-96ab-7c5472a3a16f admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.773542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1f0c6-1c8c-4c8c-96ab-7c5472a3a16f admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.773829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1f0c6-1c8c-4c8c-96ab-7c5472a3a16f admin admin] PATH_INFO: `/v3/projects/98b61a648ed24c8b8caeb4e31475395d/users/f0cf95af1cbb48f49ee2a20d64f693b4/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.774972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcdad0e-d940-4aa4-acfc-d79caddb7119 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:34.790319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5dcdad0e-d940-4aa4-acfc-d79caddb7119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:34.790514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:34.796875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1f0c6-1c8c-4c8c-96ab-7c5472a3a16f admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b61a648ed24c8b8caeb4e31475395d, user_id=f0cf95af1cbb48f49ee2a20d64f693b4, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:34.799744 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1f0c6-1c8c-4c8c-96ab-7c5472a3a16f admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:34.815317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:34.845921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcdad0e-d940-4aa4-acfc-d79caddb7119 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:34.849760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e080f-81dd-4376-9947-9acd5ba9dcf4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28954d67689a46ee975fdfc9139e41eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:34.850448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e080f-81dd-4376-9947-9acd5ba9dcf4 service glance] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.850741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e080f-81dd-4376-9947-9acd5ba9dcf4 service glance] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.851023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e080f-81dd-4376-9947-9acd5ba9dcf4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.862397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e080f-81dd-4376-9947-9acd5ba9dcf4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:34.863840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e080f-81dd-4376-9947-9acd5ba9dcf4 service glance] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:34.875363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1172/2332] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:12:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:12:34.885288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1161/2333] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:12:34 2022] PUT /identity/v3/projects/98b61a648ed24c8b8caeb4e31475395d/users/f0cf95af1cbb48f49ee2a20d64f693b4/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:12:34.885487 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e080f-81dd-4376-9947-9acd5ba9dcf4 service glance] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:34.894311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e080f-81dd-4376-9947-9acd5ba9dcf4 service glance] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:34.958320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87aa48-0004-42fa-be0b-54c38466a909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:34.959022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87aa48-0004-42fa-be0b-54c38466a909 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.959308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87aa48-0004-42fa-be0b-54c38466a909 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.959587 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87aa48-0004-42fa-be0b-54c38466a909 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.960310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87aa48-0004-42fa-be0b-54c38466a909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:34.961908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87aa48-0004-42fa-be0b-54c38466a909 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:34.977572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1173/2334] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:12:34.978821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1f0c6-1c8c-4c8c-96ab-7c5472a3a16f admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:34.990501 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1f0c6-1c8c-4c8c-96ab-7c5472a3a16f admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:34.996253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd29e94-69a0-4269-9fe1-e18d3daefcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:34.997156 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd29e94-69a0-4269-9fe1-e18d3daefcfc admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.997439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd29e94-69a0-4269-9fe1-e18d3daefcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.997712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd29e94-69a0-4269-9fe1-e18d3daefcfc admin admin] PATH_INFO: `/v3/projects/98b61a648ed24c8b8caeb4e31475395d/users/f0cf95af1cbb48f49ee2a20d64f693b4/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.999782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd29e94-69a0-4269-9fe1-e18d3daefcfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b61a648ed24c8b8caeb4e31475395d, user_id=f0cf95af1cbb48f49ee2a20d64f693b4, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:35.002765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd29e94-69a0-4269-9fe1-e18d3daefcfc admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:35.013475 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1162/2335] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:12:34 2022] PUT /identity/v3/projects/98b61a648ed24c8b8caeb4e31475395d/users/f0cf95af1cbb48f49ee2a20d64f693b4/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:12:35.158654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98d3f6-d388-4028-b350-59f817d8cbe9 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:35.158654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98d3f6-d388-4028-b350-59f817d8cbe9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:35.158654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98d3f6-d388-4028-b350-59f817d8cbe9 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:35.158654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1174/2336] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 19:12:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:12:35.164367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d59d1-43db-4084-868a-fbe4645db857 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:35.164667 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d59d1-43db-4084-868a-fbe4645db857 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:35.164953 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d59d1-43db-4084-868a-fbe4645db857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:35.186149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-cb2d59d1-43db-4084-868a-fbe4645db857 None None] MFA Rules not processed for user `ffcf280bc6d54b2fb2d22634468ac6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:35.188344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-cb2d59d1-43db-4084-868a-fbe4645db857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:35.249776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1163/2337] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 19:12:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:12:35.261859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98d3f6-d388-4028-b350-59f817d8cbe9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:35.263720 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5c98d3f6-d388-4028-b350-59f817d8cbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:35.277727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98d3f6-d388-4028-b350-59f817d8cbe9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:35.282456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c789c-fa44-4d95-b649-37d044a58db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:35.283129 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c789c-fa44-4d95-b649-37d044a58db7 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:35.283435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c789c-fa44-4d95-b649-37d044a58db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:35.283715 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c789c-fa44-4d95-b649-37d044a58db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:35.292422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c789c-fa44-4d95-b649-37d044a58db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:35.293982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c789c-fa44-4d95-b649-37d044a58db7 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:35.303973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1175/2338] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:12:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:12:35.450239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56760d3-23dc-40e1-9c80-df6ea4e10a50 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:35.450684 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56760d3-23dc-40e1-9c80-df6ea4e10a50 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:35.451033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56760d3-23dc-40e1-9c80-df6ea4e10a50 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:35.453000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1164/2339] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 19:12:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:12:35.459644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bcce0-9f31-4e93-b91f-40fa84519e29 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:35.463335 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bcce0-9f31-4e93-b91f-40fa84519e29 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:35.463335 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bcce0-9f31-4e93-b91f-40fa84519e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:35.481255 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-090bcce0-9f31-4e93-b91f-40fa84519e29 None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:35.482543 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-090bcce0-9f31-4e93-b91f-40fa84519e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:35.555121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1176/2340] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 19:12:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:12:35.568719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56760d3-23dc-40e1-9c80-df6ea4e10a50 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:35.583046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56760d3-23dc-40e1-9c80-df6ea4e10a50 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:35.586870 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f7ac2-87f3-4728-bd50-66f6293a05b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:35.587532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f7ac2-87f3-4728-bd50-66f6293a05b7 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:35.587821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f7ac2-87f3-4728-bd50-66f6293a05b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:35.588105 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f7ac2-87f3-4728-bd50-66f6293a05b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:35.600160 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f7ac2-87f3-4728-bd50-66f6293a05b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:35.601632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f7ac2-87f3-4728-bd50-66f6293a05b7 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:35.618633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1165/2341] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:12:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:12:42.075141 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77efe20-da33-426b-9067-e28d31ce68b7 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:42.075580 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77efe20-da33-426b-9067-e28d31ce68b7 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:42.075678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77efe20-da33-426b-9067-e28d31ce68b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:42.114178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-a77efe20-da33-426b-9067-e28d31ce68b7 None None] MFA Rules not processed for user `f0cf95af1cbb48f49ee2a20d64f693b4`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:42.115766 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a77efe20-da33-426b-9067-e28d31ce68b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:42.168665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:42.261515 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1177/2342] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:12:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:12:42.276775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f7ac2-87f3-4728-bd50-66f6293a05b7 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:42.279164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6d8f7ac2-87f3-4728-bd50-66f6293a05b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:42.312362 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f7ac2-87f3-4728-bd50-66f6293a05b7 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:42.327145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3119c5f-7573-4ab7-a973-02b7e78c69f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:42.327773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3119c5f-7573-4ab7-a973-02b7e78c69f0 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:42.328227 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3119c5f-7573-4ab7-a973-02b7e78c69f0 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:42.328708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3119c5f-7573-4ab7-a973-02b7e78c69f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:42.425957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3119c5f-7573-4ab7-a973-02b7e78c69f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:42.428594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3119c5f-7573-4ab7-a973-02b7e78c69f0 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:42.440701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1166/2343] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:12:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:12:42.963665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edbefc-cc7f-43ff-b850-177feff2abbc None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:42.964073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edbefc-cc7f-43ff-b850-177feff2abbc None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:42.964473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edbefc-cc7f-43ff-b850-177feff2abbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:42.985663 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-27edbefc-cc7f-43ff-b850-177feff2abbc None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:42.987133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-27edbefc-cc7f-43ff-b850-177feff2abbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:43.054598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1178/2344] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:12:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:12:43.068334 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3119c5f-7573-4ab7-a973-02b7e78c69f0 service nova] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:43.070033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b3119c5f-7573-4ab7-a973-02b7e78c69f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:43.149646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3119c5f-7573-4ab7-a973-02b7e78c69f0 service nova] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:43.155925 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68059352-b381-462e-9a0a-db4b05023a99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:43.156591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68059352-b381-462e-9a0a-db4b05023a99 service swift] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:43.156879 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68059352-b381-462e-9a0a-db4b05023a99 service swift] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:43.157162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68059352-b381-462e-9a0a-db4b05023a99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:43.164837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:43.166965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68059352-b381-462e-9a0a-db4b05023a99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:43.168634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68059352-b381-462e-9a0a-db4b05023a99 service swift] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:43.178036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1167/2345] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:12:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 19:12:46.031706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49c291-62fa-46b0-99f0-551cf960cba4 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:46.032287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49c291-62fa-46b0-99f0-551cf960cba4 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:46.032287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49c291-62fa-46b0-99f0-551cf960cba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:46.063951 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ee49c291-62fa-46b0-99f0-551cf960cba4 None None] MFA Rules not processed for user `9631ec5df93c42df9d2f60bedfef6507`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:46.065821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ee49c291-62fa-46b0-99f0-551cf960cba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:46.091679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1179/2346] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:12:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:12:53.335561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:56.057007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=43 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:20.684975 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca24cd-ddab-4894-b030-05bbcc301aa8 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:20.685519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca24cd-ddab-4894-b030-05bbcc301aa8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:20.685639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca24cd-ddab-4894-b030-05bbcc301aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:20.707616 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-bcca24cd-ddab-4894-b030-05bbcc301aa8 None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:13:20.708831 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bcca24cd-ddab-4894-b030-05bbcc301aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:13:20.716457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1168/2347] 10.222.0.17 () {58 vars in 967 bytes} [Mon Jul 25 19:13:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:13:20.739035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49c291-62fa-46b0-99f0-551cf960cba4 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:20.750908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49c291-62fa-46b0-99f0-551cf960cba4 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:20.756617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15adcf7-f9c2-4654-8dcf-6e2160cdee8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:20.757267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15adcf7-f9c2-4654-8dcf-6e2160cdee8b service swift] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:20.757555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15adcf7-f9c2-4654-8dcf-6e2160cdee8b service swift] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:20.757835 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15adcf7-f9c2-4654-8dcf-6e2160cdee8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:20.766254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15adcf7-f9c2-4654-8dcf-6e2160cdee8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:20.767891 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15adcf7-f9c2-4654-8dcf-6e2160cdee8b service swift] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:20.776818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1180/2348] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:13:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:13:30.701718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:30.773299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=6 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:36.488324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453d595-78be-4728-a980-21dd095afa87 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:36.488898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453d595-78be-4728-a980-21dd095afa87 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:36.489223 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453d595-78be-4728-a980-21dd095afa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:36.509519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-d453d595-78be-4728-a980-21dd095afa87 None None] MFA Rules not processed for user `7e3f168283ba4f159948830f396ae7cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:13:36.510958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d453d595-78be-4728-a980-21dd095afa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:13:36.570539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1169/2349] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:13:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:13:43.047541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15adcf7-f9c2-4654-8dcf-6e2160cdee8b service swift] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:43.049638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e15adcf7-f9c2-4654-8dcf-6e2160cdee8b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:13:43.068559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15adcf7-f9c2-4654-8dcf-6e2160cdee8b service swift] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:43.074286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d6292-ec04-4202-ba19-d677d8d374bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:43.075003 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d6292-ec04-4202-ba19-d677d8d374bb service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:43.075274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d6292-ec04-4202-ba19-d677d8d374bb service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:43.075553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d6292-ec04-4202-ba19-d677d8d374bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:43.077364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fd0d6292-ec04-4202-ba19-d677d8d374bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:13:43.090918 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d6292-ec04-4202-ba19-d677d8d374bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:43.092533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d6292-ec04-4202-ba19-d677d8d374bb service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:43.110301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1181/2350] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:13:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:13:44.336547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453d595-78be-4728-a980-21dd095afa87 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:44.350909 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453d595-78be-4728-a980-21dd095afa87 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:44.354866 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d20f5-1ed7-4291-806c-85ac9d5f2389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:44.355644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d20f5-1ed7-4291-806c-85ac9d5f2389 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:44.355995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d20f5-1ed7-4291-806c-85ac9d5f2389 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:44.356341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d20f5-1ed7-4291-806c-85ac9d5f2389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:44.383831 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d20f5-1ed7-4291-806c-85ac9d5f2389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:44.385891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d20f5-1ed7-4291-806c-85ac9d5f2389 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:44.400416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1170/2351] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:13:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:13:53.105420 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=6 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:54.393298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=31 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:29.758711 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d6292-ec04-4202-ba19-d677d8d374bb service nova] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:14:29.770018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d6292-ec04-4202-ba19-d677d8d374bb service nova] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:14:29.775779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477fc0d9-ea5c-4a3b-834b-e95eba86aadd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:14:29.776428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fc0d9-ea5c-4a3b-834b-e95eba86aadd service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:29.776718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fc0d9-ea5c-4a3b-834b-e95eba86aadd service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:29.777002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fc0d9-ea5c-4a3b-834b-e95eba86aadd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:29.778863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-477fc0d9-ea5c-4a3b-834b-e95eba86aadd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:29.862708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477fc0d9-ea5c-4a3b-834b-e95eba86aadd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:14:29.864762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477fc0d9-ea5c-4a3b-834b-e95eba86aadd service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:14:29.885679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1182/2352] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:14:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:14:39.882857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=41 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:12.915824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d20f5-1ed7-4291-806c-85ac9d5f2389 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:12.926580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d20f5-1ed7-4291-806c-85ac9d5f2389 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:12.930414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648247d-9847-46d0-8581-abe12fbda008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:12.931060 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648247d-9847-46d0-8581-abe12fbda008 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:12.931361 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648247d-9847-46d0-8581-abe12fbda008 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:12.931621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648247d-9847-46d0-8581-abe12fbda008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:12.933607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2648247d-9847-46d0-8581-abe12fbda008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:12.944251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648247d-9847-46d0-8581-abe12fbda008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:12.945822 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648247d-9847-46d0-8581-abe12fbda008 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:12.954077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1171/2353] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:15:15.688811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477fc0d9-ea5c-4a3b-834b-e95eba86aadd service nova] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:15.699956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477fc0d9-ea5c-4a3b-834b-e95eba86aadd service nova] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:15.703614 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c5589-02a7-492d-b4dc-5bced2bf5439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:15.704267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5589-02a7-492d-b4dc-5bced2bf5439 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:15.704583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5589-02a7-492d-b4dc-5bced2bf5439 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:15.704867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5589-02a7-492d-b4dc-5bced2bf5439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:15.706705 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8f4c5589-02a7-492d-b4dc-5bced2bf5439 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:15.720837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c5589-02a7-492d-b4dc-5bced2bf5439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:15.722485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c5589-02a7-492d-b4dc-5bced2bf5439 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:15.734317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1183/2354] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:15:22.950532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=6 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:24.005793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0db234-cebe-413d-8b06-a7c882016b05 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:24.006452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0db234-cebe-413d-8b06-a7c882016b05 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:24.006452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0db234-cebe-413d-8b06-a7c882016b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:24.059383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-cc0db234-cebe-413d-8b06-a7c882016b05 None None] MFA Rules not processed for user `c99e02a845184f0aa206ba9f209ae4ef`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:24.060607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-cc0db234-cebe-413d-8b06-a7c882016b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:24.132023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1172/2355] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:15:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:24.463797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c5589-02a7-492d-b4dc-5bced2bf5439 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:24.474905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c5589-02a7-492d-b4dc-5bced2bf5439 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:24.478985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129a2d5-a55d-458f-81ac-1b1beec64f54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:24.479649 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129a2d5-a55d-458f-81ac-1b1beec64f54 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:24.479938 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129a2d5-a55d-458f-81ac-1b1beec64f54 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:24.480216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129a2d5-a55d-458f-81ac-1b1beec64f54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:24.482234 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f129a2d5-a55d-458f-81ac-1b1beec64f54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:24.493840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129a2d5-a55d-458f-81ac-1b1beec64f54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:24.495342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129a2d5-a55d-458f-81ac-1b1beec64f54 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:24.506104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1184/2356] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:15:27.162372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0db234-cebe-413d-8b06-a7c882016b05 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:27.164136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-cc0db234-cebe-413d-8b06-a7c882016b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:27.246949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0db234-cebe-413d-8b06-a7c882016b05 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:27.251052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d645d-0575-43ff-8170-e54d230ebd70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:27.251733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d645d-0575-43ff-8170-e54d230ebd70 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:27.252029 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d645d-0575-43ff-8170-e54d230ebd70 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:27.252406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d645d-0575-43ff-8170-e54d230ebd70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:27.254255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ef8d645d-0575-43ff-8170-e54d230ebd70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:27.276307 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d645d-0575-43ff-8170-e54d230ebd70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:27.279134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d645d-0575-43ff-8170-e54d230ebd70 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:27.303032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1173/2357] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:15:29.180565 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129a2d5-a55d-458f-81ac-1b1beec64f54 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:29.199409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129a2d5-a55d-458f-81ac-1b1beec64f54 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:29.205717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c4de6-f89b-4517-92d8-577a46150db6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:29.206107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c4de6-f89b-4517-92d8-577a46150db6 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:29.206439 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c4de6-f89b-4517-92d8-577a46150db6 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:29.206729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c4de6-f89b-4517-92d8-577a46150db6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:29.208667 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1a5c4de6-f89b-4517-92d8-577a46150db6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:29.223380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c4de6-f89b-4517-92d8-577a46150db6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:29.225017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c4de6-f89b-4517-92d8-577a46150db6 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:29.239510 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1185/2358] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:15:37.294131 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=67 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:39.232004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=18 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:51.420435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d645d-0575-43ff-8170-e54d230ebd70 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:51.435072 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d645d-0575-43ff-8170-e54d230ebd70 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:51.441348 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb176418-65ed-4a10-9c34-efbce79c2a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:51.442032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb176418-65ed-4a10-9c34-efbce79c2a66 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:51.442348 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb176418-65ed-4a10-9c34-efbce79c2a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:51.442629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb176418-65ed-4a10-9c34-efbce79c2a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:51.444665 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bb176418-65ed-4a10-9c34-efbce79c2a66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:51.517188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb176418-65ed-4a10-9c34-efbce79c2a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:51.518668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb176418-65ed-4a10-9c34-efbce79c2a66 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:51.528675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1174/2359] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:15:55.680709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c4de6-f89b-4517-92d8-577a46150db6 service nova] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:55.682757 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1a5c4de6-f89b-4517-92d8-577a46150db6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:55.698134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c4de6-f89b-4517-92d8-577a46150db6 service nova] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:55.703282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752aed0-acd0-435d-bba2-16367172a2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:55.704355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752aed0-acd0-435d-bba2-16367172a2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:55.704816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752aed0-acd0-435d-bba2-16367172a2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:55.705138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752aed0-acd0-435d-bba2-16367172a2f9 admin admin] PATH_INFO: `/v3/users/7e3f168283ba4f159948830f396ae7cf` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:55.721416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752aed0-acd0-435d-bba2-16367172a2f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e3f168283ba4f159948830f396ae7cf)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:55.726601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752aed0-acd0-435d-bba2-16367172a2f9 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:55.930844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-c752aed0-acd0-435d-bba2-16367172a2f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e3f168283ba4f159948830f396ae7cf'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:55.931519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-c752aed0-acd0-435d-bba2-16367172a2f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e3f168283ba4f159948830f396ae7cf'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:55.939878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-c752aed0-acd0-435d-bba2-16367172a2f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e3f168283ba4f159948830f396ae7cf'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:55.954599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-c752aed0-acd0-435d-bba2-16367172a2f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e3f168283ba4f159948830f396ae7cf'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:56.053398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-c752aed0-acd0-435d-bba2-16367172a2f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e3f168283ba4f159948830f396ae7cf'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:56.059984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1186/2360] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:15:55 2022] DELETE /identity/v3/users/7e3f168283ba4f159948830f396ae7cf => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:15:56.576091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb176418-65ed-4a10-9c34-efbce79c2a66 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:56.591219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb176418-65ed-4a10-9c34-efbce79c2a66 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:56.655026 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9603bc-8f55-4612-9d16-c71327ba6b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:56.655889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9603bc-8f55-4612-9d16-c71327ba6b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:56.656230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9603bc-8f55-4612-9d16-c71327ba6b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:56.656499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9603bc-8f55-4612-9d16-c71327ba6b6e admin admin] PATH_INFO: `/v3/projects/08c6be8edb884214bae7a0abbf97dc16` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:56.667972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9603bc-8f55-4612-9d16-c71327ba6b6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c6be8edb884214bae7a0abbf97dc16)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:56.669899 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9603bc-8f55-4612-9d16-c71327ba6b6e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:56.788289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ec9603bc-8f55-4612-9d16-c71327ba6b6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c6be8edb884214bae7a0abbf97dc16', 'request_id': 'req-ec9603bc-8f55-4612-9d16-c71327ba6b6e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:56.861758 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ec9603bc-8f55-4612-9d16-c71327ba6b6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c6be8edb884214bae7a0abbf97dc16', 'request_id': 'req-ec9603bc-8f55-4612-9d16-c71327ba6b6e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:56.868334 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ec9603bc-8f55-4612-9d16-c71327ba6b6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c6be8edb884214bae7a0abbf97dc16', 'request_id': 'req-ec9603bc-8f55-4612-9d16-c71327ba6b6e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:56.881140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ec9603bc-8f55-4612-9d16-c71327ba6b6e admin admin] The token cache is being invalidate because project 08c6be8edb884214bae7a0abbf97dc16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:15:56.883174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ec9603bc-8f55-4612-9d16-c71327ba6b6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:56.883174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ec9603bc-8f55-4612-9d16-c71327ba6b6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:56.883601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1175/2361] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:15:56 2022] DELETE /identity/v3/projects/08c6be8edb884214bae7a0abbf97dc16 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:15:56.888626 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92524ee4-8c30-45b8-aaa1-e425386b6cdf None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:56.889131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92524ee4-8c30-45b8-aaa1-e425386b6cdf None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:56.889280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92524ee4-8c30-45b8-aaa1-e425386b6cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:56.910548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-92524ee4-8c30-45b8-aaa1-e425386b6cdf None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:56.911933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-92524ee4-8c30-45b8-aaa1-e425386b6cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:56.973896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1187/2362] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:56.982587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8534ddf-a873-4a30-9096-30a0e71295bb None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:56.982885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8534ddf-a873-4a30-9096-30a0e71295bb None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:56.983174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8534ddf-a873-4a30-9096-30a0e71295bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:57.004533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c8534ddf-a873-4a30-9096-30a0e71295bb None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:57.005679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c8534ddf-a873-4a30-9096-30a0e71295bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:57.013896 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1176/2363] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:57.020326 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92524ee4-8c30-45b8-aaa1-e425386b6cdf None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:57.029136 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92524ee4-8c30-45b8-aaa1-e425386b6cdf None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:57.034053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9646e7d-d7ec-4bff-984e-10d7fb65fe65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:57.034773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9646e7d-d7ec-4bff-984e-10d7fb65fe65 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:57.035061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9646e7d-d7ec-4bff-984e-10d7fb65fe65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:57.035344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9646e7d-d7ec-4bff-984e-10d7fb65fe65 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:57.036084 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9646e7d-d7ec-4bff-984e-10d7fb65fe65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:15:57.036395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9646e7d-d7ec-4bff-984e-10d7fb65fe65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:57.038015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9646e7d-d7ec-4bff-984e-10d7fb65fe65 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:57.050377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1188/2364] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:15:57.054727 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8534ddf-a873-4a30-9096-30a0e71295bb None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:57.065148 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8534ddf-a873-4a30-9096-30a0e71295bb None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:57.069869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37632db0-60c8-4733-945d-794054bd1113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:57.070567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37632db0-60c8-4733-945d-794054bd1113 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:57.070859 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37632db0-60c8-4733-945d-794054bd1113 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:57.071135 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37632db0-60c8-4733-945d-794054bd1113 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:57.071823 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37632db0-60c8-4733-945d-794054bd1113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:57.073723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37632db0-60c8-4733-945d-794054bd1113 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:57.080436 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:57.210961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1177/2365] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:15:57 2022] POST /identity/v3/projects => generated 403 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:15:57.216175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9646e7d-d7ec-4bff-984e-10d7fb65fe65 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:57.227115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9646e7d-d7ec-4bff-984e-10d7fb65fe65 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:57.275980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:57.293279 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6b11f-f3ff-4520-bd2b-0c6fe1987e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:57.293962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6b11f-f3ff-4520-bd2b-0c6fe1987e41 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:57.294247 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6b11f-f3ff-4520-bd2b-0c6fe1987e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:57.294550 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6b11f-f3ff-4520-bd2b-0c6fe1987e41 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:57.295245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6b11f-f3ff-4520-bd2b-0c6fe1987e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:57.297204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6b11f-f3ff-4520-bd2b-0c6fe1987e41 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:57.388709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1189/2366] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:15:57 2022] POST /identity/v3/users => generated 302 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:15:57.393550 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37632db0-60c8-4733-945d-794054bd1113 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:57.404336 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37632db0-60c8-4733-945d-794054bd1113 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:57.409659 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc395ee8-0775-49cb-af8f-652f2edf94c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:57.410349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc395ee8-0775-49cb-af8f-652f2edf94c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:57.410655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc395ee8-0775-49cb-af8f-652f2edf94c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:57.410938 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc395ee8-0775-49cb-af8f-652f2edf94c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:57.411760 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc395ee8-0775-49cb-af8f-652f2edf94c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:57.413380 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc395ee8-0775-49cb-af8f-652f2edf94c8 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:57.425676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1178/2367] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:15:57.429639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6b11f-f3ff-4520-bd2b-0c6fe1987e41 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:57.441093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6b11f-f3ff-4520-bd2b-0c6fe1987e41 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:57.450405 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd67cc1-c3f6-4e17-87f9-f357a30cb545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:57.450405 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd67cc1-c3f6-4e17-87f9-f357a30cb545 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:57.450405 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd67cc1-c3f6-4e17-87f9-f357a30cb545 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:57.450405 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd67cc1-c3f6-4e17-87f9-f357a30cb545 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:57.450845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd67cc1-c3f6-4e17-87f9-f357a30cb545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:57.450845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd67cc1-c3f6-4e17-87f9-f357a30cb545 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:57.461607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1190/2368] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:15:57.466065 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc395ee8-0775-49cb-af8f-652f2edf94c8 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:57.477179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc395ee8-0775-49cb-af8f-652f2edf94c8 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:57.482049 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ae3c0-2fea-43cb-b5d4-74a2093b6132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:57.483008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ae3c0-2fea-43cb-b5d4-74a2093b6132 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:57.483300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ae3c0-2fea-43cb-b5d4-74a2093b6132 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:57.483612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ae3c0-2fea-43cb-b5d4-74a2093b6132 admin admin] PATH_INFO: `/v3/projects/c0ee114a01124d1ebef76f1e2007ac3b/users/923a4f0309244ac1b462ef9b925e7296/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:57.499767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ae3c0-2fea-43cb-b5d4-74a2093b6132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ee114a01124d1ebef76f1e2007ac3b, user_id=923a4f0309244ac1b462ef9b925e7296, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:57.503131 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ae3c0-2fea-43cb-b5d4-74a2093b6132 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:57.608083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1179/2369] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:15:57 2022] PUT /identity/v3/projects/c0ee114a01124d1ebef76f1e2007ac3b/users/923a4f0309244ac1b462ef9b925e7296/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:15:57.618546 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614d590-408b-44a7-bc64-c9c7be5998ef None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:57.618839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614d590-408b-44a7-bc64-c9c7be5998ef None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:57.619157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614d590-408b-44a7-bc64-c9c7be5998ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:57.647576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-9614d590-408b-44a7-bc64-c9c7be5998ef None None] MFA Rules not processed for user `923a4f0309244ac1b462ef9b925e7296`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:57.648715 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9614d590-408b-44a7-bc64-c9c7be5998ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:57.716745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1191/2370] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:15:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:57.722294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ae3c0-2fea-43cb-b5d4-74a2093b6132 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:57.787086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ae3c0-2fea-43cb-b5d4-74a2093b6132 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:57.793224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fdafa-12f1-4be6-a57d-33b37ab0664f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:57.793934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fdafa-12f1-4be6-a57d-33b37ab0664f admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:57.794273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fdafa-12f1-4be6-a57d-33b37ab0664f admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:57.794584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fdafa-12f1-4be6-a57d-33b37ab0664f admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:57.795281 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fdafa-12f1-4be6-a57d-33b37ab0664f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:57.797496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fdafa-12f1-4be6-a57d-33b37ab0664f admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:57.980805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1180/2371] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:15:57 2022] POST /identity/v3/projects => generated 403 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:15:57.989738 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614d590-408b-44a7-bc64-c9c7be5998ef None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:58.002971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614d590-408b-44a7-bc64-c9c7be5998ef None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:58.064587 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c1c4d-638b-4892-bb4e-21aae0b0af14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:58.064999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c1c4d-638b-4892-bb4e-21aae0b0af14 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:58.065289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c1c4d-638b-4892-bb4e-21aae0b0af14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:58.065571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c1c4d-638b-4892-bb4e-21aae0b0af14 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:58.066391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c1c4d-638b-4892-bb4e-21aae0b0af14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:58.068430 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c1c4d-638b-4892-bb4e-21aae0b0af14 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:58.198889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1192/2372] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:15:57 2022] POST /identity/v3/users => generated 308 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:15:58.202844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fdafa-12f1-4be6-a57d-33b37ab0664f admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:58.213904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fdafa-12f1-4be6-a57d-33b37ab0664f admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:58.218515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690c060-44be-46dc-99c1-c6ef158a9c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:58.219224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690c060-44be-46dc-99c1-c6ef158a9c60 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:58.219539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690c060-44be-46dc-99c1-c6ef158a9c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:58.219821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690c060-44be-46dc-99c1-c6ef158a9c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:58.220546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690c060-44be-46dc-99c1-c6ef158a9c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:58.222149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690c060-44be-46dc-99c1-c6ef158a9c60 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:58.233333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1181/2373] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:15:58.238480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c1c4d-638b-4892-bb4e-21aae0b0af14 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:58.249236 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c1c4d-638b-4892-bb4e-21aae0b0af14 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:58.254891 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde350f1-e940-40e6-8308-50740385914b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:58.255855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde350f1-e940-40e6-8308-50740385914b admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:58.256083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde350f1-e940-40e6-8308-50740385914b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:58.256363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde350f1-e940-40e6-8308-50740385914b admin admin] PATH_INFO: `/v3/projects/da9d800e8c874053b3452266752a5173/users/c8a6210c65ad427aabbdc6ce5ca8dc06/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:58.273942 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde350f1-e940-40e6-8308-50740385914b admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9d800e8c874053b3452266752a5173, user_id=c8a6210c65ad427aabbdc6ce5ca8dc06, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:58.276653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde350f1-e940-40e6-8308-50740385914b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:58.479393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1193/2374] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:15:58 2022] PUT /identity/v3/projects/da9d800e8c874053b3452266752a5173/users/c8a6210c65ad427aabbdc6ce5ca8dc06/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:15:58.483471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690c060-44be-46dc-99c1-c6ef158a9c60 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:58.494590 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690c060-44be-46dc-99c1-c6ef158a9c60 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:58.556873 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8d661-af43-41ea-955f-24e3c14d6827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:58.557556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8d661-af43-41ea-955f-24e3c14d6827 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:58.557851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8d661-af43-41ea-955f-24e3c14d6827 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:58.558136 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8d661-af43-41ea-955f-24e3c14d6827 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:58.558905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8d661-af43-41ea-955f-24e3c14d6827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:58.560509 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8d661-af43-41ea-955f-24e3c14d6827 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:58.570855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1182/2375] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:15:58.575249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde350f1-e940-40e6-8308-50740385914b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:58.586184 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde350f1-e940-40e6-8308-50740385914b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:58.591149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900a658-aafa-43d7-be81-ce84e2336836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:58.592043 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900a658-aafa-43d7-be81-ce84e2336836 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:58.592333 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900a658-aafa-43d7-be81-ce84e2336836 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:58.592693 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900a658-aafa-43d7-be81-ce84e2336836 admin admin] PATH_INFO: `/v3/projects/da9d800e8c874053b3452266752a5173/users/c8a6210c65ad427aabbdc6ce5ca8dc06/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:58.595363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900a658-aafa-43d7-be81-ce84e2336836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9d800e8c874053b3452266752a5173, user_id=c8a6210c65ad427aabbdc6ce5ca8dc06, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:58.598019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900a658-aafa-43d7-be81-ce84e2336836 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:58.608820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1194/2376] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:15:58 2022] PUT /identity/v3/projects/da9d800e8c874053b3452266752a5173/users/c8a6210c65ad427aabbdc6ce5ca8dc06/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:15:58.621291 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37a038-07d4-4fdd-976b-2eb35a05a5a4 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:58.621855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37a038-07d4-4fdd-976b-2eb35a05a5a4 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:58.621994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37a038-07d4-4fdd-976b-2eb35a05a5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:58.650153 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-6a37a038-07d4-4fdd-976b-2eb35a05a5a4 None None] MFA Rules not processed for user `c8a6210c65ad427aabbdc6ce5ca8dc06`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:58.651468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6a37a038-07d4-4fdd-976b-2eb35a05a5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:58.661921 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:58.721903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1183/2377] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:15:58 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:58.732644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900a658-aafa-43d7-be81-ce84e2336836 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:58.734389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b900a658-aafa-43d7-be81-ce84e2336836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:58.746947 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:58.809686 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900a658-aafa-43d7-be81-ce84e2336836 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:58.813911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8f72a-59fc-4d62-82c0-09991ddc416d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:58.814559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8f72a-59fc-4d62-82c0-09991ddc416d service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:58.814985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8f72a-59fc-4d62-82c0-09991ddc416d service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:58.815112 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8f72a-59fc-4d62-82c0-09991ddc416d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:58.831971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8f72a-59fc-4d62-82c0-09991ddc416d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:58.833307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8f72a-59fc-4d62-82c0-09991ddc416d service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:58.846210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1195/2378] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:58 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:15:58.986700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37a038-07d4-4fdd-976b-2eb35a05a5a4 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:59.051599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37a038-07d4-4fdd-976b-2eb35a05a5a4 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:59.056581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:59.057470 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:59.057809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:59.058085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] PATH_INFO: `/v3/users/923a4f0309244ac1b462ef9b925e7296` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:59.059536 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=923a4f0309244ac1b462ef9b925e7296)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:59.061585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:59.373287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '923a4f0309244ac1b462ef9b925e7296'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:59.373912 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '923a4f0309244ac1b462ef9b925e7296'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:59.374565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '923a4f0309244ac1b462ef9b925e7296'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:59.464834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '923a4f0309244ac1b462ef9b925e7296'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:59.478583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '923a4f0309244ac1b462ef9b925e7296'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:15:59.492036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1184/2379] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:15:58 2022] DELETE /identity/v3/users/923a4f0309244ac1b462ef9b925e7296 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:15:59.497158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3b28e-bbf0-4c1f-a266-abb27db5f377 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:59.502536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3b28e-bbf0-4c1f-a266-abb27db5f377 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:59.502536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3b28e-bbf0-4c1f-a266-abb27db5f377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:59.516831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-dfc3b28e-bbf0-4c1f-a266-abb27db5f377 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:59.518219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-dfc3b28e-bbf0-4c1f-a266-abb27db5f377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:59.582664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1196/2380] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:59.592713 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:59.594838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:59.628025 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:59.658178 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfae92-2c59-4965-8adb-8462550b4e4c admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:59.662785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61fb04-8cf7-419f-8443-efd8f431613f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:59.663468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61fb04-8cf7-419f-8443-efd8f431613f service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:59.663753 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61fb04-8cf7-419f-8443-efd8f431613f service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:59.664037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61fb04-8cf7-419f-8443-efd8f431613f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:59.674306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61fb04-8cf7-419f-8443-efd8f431613f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:59.675844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61fb04-8cf7-419f-8443-efd8f431613f service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:59.685718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1185/2381] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:00.114819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3b28e-bbf0-4c1f-a266-abb27db5f377 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:00.119913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-dfc3b28e-bbf0-4c1f-a266-abb27db5f377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:00.130876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3b28e-bbf0-4c1f-a266-abb27db5f377 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:00.143464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:00.144313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:00.144603 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:00.144919 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] PATH_INFO: `/v3/projects/c0ee114a01124d1ebef76f1e2007ac3b` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:00.146088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ee114a01124d1ebef76f1e2007ac3b)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:00.148118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:00.274050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ee114a01124d1ebef76f1e2007ac3b', 'request_id': 'req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:00.280397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ee114a01124d1ebef76f1e2007ac3b', 'request_id': 'req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:00.286158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ee114a01124d1ebef76f1e2007ac3b', 'request_id': 'req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:00.382361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] The token cache is being invalidate because project c0ee114a01124d1ebef76f1e2007ac3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:00.382672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:00.383377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:00.385017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1197/2382] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:00 2022] DELETE /identity/v3/projects/c0ee114a01124d1ebef76f1e2007ac3b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:00.389113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61fb04-8cf7-419f-8443-efd8f431613f service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:00.401223 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61fb04-8cf7-419f-8443-efd8f431613f service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:00.463936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5529377-4aec-40b2-9645-7a94ddaa2e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:00.464826 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5529377-4aec-40b2-9645-7a94ddaa2e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:00.465113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5529377-4aec-40b2-9645-7a94ddaa2e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:00.465396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5529377-4aec-40b2-9645-7a94ddaa2e6d admin admin] PATH_INFO: `/v3/users/c8a6210c65ad427aabbdc6ce5ca8dc06` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:00.466624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5529377-4aec-40b2-9645-7a94ddaa2e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a6210c65ad427aabbdc6ce5ca8dc06)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:00.468551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5529377-4aec-40b2-9645-7a94ddaa2e6d admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:00.693575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f5529377-4aec-40b2-9645-7a94ddaa2e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a6210c65ad427aabbdc6ce5ca8dc06'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:00.694270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f5529377-4aec-40b2-9645-7a94ddaa2e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a6210c65ad427aabbdc6ce5ca8dc06'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:00.694918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f5529377-4aec-40b2-9645-7a94ddaa2e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a6210c65ad427aabbdc6ce5ca8dc06'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:00.786723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f5529377-4aec-40b2-9645-7a94ddaa2e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a6210c65ad427aabbdc6ce5ca8dc06'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:00.802498 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f5529377-4aec-40b2-9645-7a94ddaa2e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a6210c65ad427aabbdc6ce5ca8dc06'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:00.815594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1186/2383] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:00 2022] DELETE /identity/v3/users/c8a6210c65ad427aabbdc6ce5ca8dc06 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:01.314694 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:01.314694 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:01.327661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda692a5-0dd3-420f-9a3a-5f308b99d6cd admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:01.367626 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:01.397408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:01.398185 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:01.398507 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:01.400613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5 admin admin] PATH_INFO: `/v3/projects/da9d800e8c874053b3452266752a5173` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:01.400954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da9d800e8c874053b3452266752a5173)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:01.403393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:01.539477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da9d800e8c874053b3452266752a5173', 'request_id': 'req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:01.546691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da9d800e8c874053b3452266752a5173', 'request_id': 'req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:01.554637 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da9d800e8c874053b3452266752a5173', 'request_id': 'req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:01.672106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5 admin admin] The token cache is being invalidate because project da9d800e8c874053b3452266752a5173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:01.672330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:01.673100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ddb595bd-f5e0-4f1c-9430-d6950d96fbf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:01.674894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1198/2384] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:01 2022] DELETE /identity/v3/projects/da9d800e8c874053b3452266752a5173 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:01.680383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909e7f5-8da0-4160-aa4c-569299d65a50 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:01.680687 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909e7f5-8da0-4160-aa4c-569299d65a50 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:01.680978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909e7f5-8da0-4160-aa4c-569299d65a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:01.701354 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-8909e7f5-8da0-4160-aa4c-569299d65a50 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:01.702633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8909e7f5-8da0-4160-aa4c-569299d65a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:01.750931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:01.767038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1187/2385] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:01.778617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb580f5-7111-4e6e-92a1-54af57555668 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:01.779090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb580f5-7111-4e6e-92a1-54af57555668 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:01.779622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb580f5-7111-4e6e-92a1-54af57555668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:01.804899 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-cfb580f5-7111-4e6e-92a1-54af57555668 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:01.806173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-cfb580f5-7111-4e6e-92a1-54af57555668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:01.815982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1199/2386] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:01.820605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909e7f5-8da0-4160-aa4c-569299d65a50 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:01.831299 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909e7f5-8da0-4160-aa4c-569299d65a50 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:01.836035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a07ca4-efc1-4f65-8779-31e6174d73ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:01.836708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a07ca4-efc1-4f65-8779-31e6174d73ca admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:01.837007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a07ca4-efc1-4f65-8779-31e6174d73ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:01.837288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a07ca4-efc1-4f65-8779-31e6174d73ca admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:01.838057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a07ca4-efc1-4f65-8779-31e6174d73ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:16:01.838399 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a07ca4-efc1-4f65-8779-31e6174d73ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:01.840074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a07ca4-efc1-4f65-8779-31e6174d73ca admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:01.853091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1188/2387] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:16:01.857109 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb580f5-7111-4e6e-92a1-54af57555668 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:01.867502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb580f5-7111-4e6e-92a1-54af57555668 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:01.872779 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada1076-e12d-4ee9-9b6a-51a6d92b2fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:01.873474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada1076-e12d-4ee9-9b6a-51a6d92b2fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:01.873768 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada1076-e12d-4ee9-9b6a-51a6d92b2fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:01.874037 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada1076-e12d-4ee9-9b6a-51a6d92b2fd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:01.874786 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada1076-e12d-4ee9-9b6a-51a6d92b2fd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:01.877227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada1076-e12d-4ee9-9b6a-51a6d92b2fd1 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:02.011355 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1200/2388] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:01 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:02.015884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a07ca4-efc1-4f65-8779-31e6174d73ca admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:02.026084 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a07ca4-efc1-4f65-8779-31e6174d73ca admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:02.083935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b007c-6684-4588-b45c-d0bee9c77ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:02.084634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b007c-6684-4588-b45c-d0bee9c77ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:02.084964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b007c-6684-4588-b45c-d0bee9c77ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:02.086021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b007c-6684-4588-b45c-d0bee9c77ab9 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:02.086021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b007c-6684-4588-b45c-d0bee9c77ab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:02.087958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b007c-6684-4588-b45c-d0bee9c77ab9 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:02.179604 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1189/2389] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:02 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:16:02.183890 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada1076-e12d-4ee9-9b6a-51a6d92b2fd1 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:02.195253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada1076-e12d-4ee9-9b6a-51a6d92b2fd1 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:02.200485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b413f-7f76-4952-9139-f52b5638d1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:02.201377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b413f-7f76-4952-9139-f52b5638d1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:02.201568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b413f-7f76-4952-9139-f52b5638d1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:02.201871 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b413f-7f76-4952-9139-f52b5638d1a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:02.202654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b413f-7f76-4952-9139-f52b5638d1a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:02.205170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b413f-7f76-4952-9139-f52b5638d1a3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:02.216424 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1201/2390] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:16:02.221619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b007c-6684-4588-b45c-d0bee9c77ab9 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:02.232149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b007c-6684-4588-b45c-d0bee9c77ab9 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:02.238177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59942ef0-6653-435e-b1d1-84b7aaa2e7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:02.239020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59942ef0-6653-435e-b1d1-84b7aaa2e7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:02.239314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59942ef0-6653-435e-b1d1-84b7aaa2e7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:02.239596 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59942ef0-6653-435e-b1d1-84b7aaa2e7f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:02.240363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59942ef0-6653-435e-b1d1-84b7aaa2e7f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:02.241968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59942ef0-6653-435e-b1d1-84b7aaa2e7f5 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:02.253885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1190/2391] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:02.258147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b413f-7f76-4952-9139-f52b5638d1a3 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:02.268954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b413f-7f76-4952-9139-f52b5638d1a3 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:02.273502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf264563-5c5c-4d71-9606-bc53a3570d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:02.274421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf264563-5c5c-4d71-9606-bc53a3570d7e admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:02.274708 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf264563-5c5c-4d71-9606-bc53a3570d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:02.274987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf264563-5c5c-4d71-9606-bc53a3570d7e admin admin] PATH_INFO: `/v3/projects/e61a5d896efe44d2a41c4088247885a4/users/f1462d1f526745aabf69e60f82e8ba8d/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:02.290884 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf264563-5c5c-4d71-9606-bc53a3570d7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e61a5d896efe44d2a41c4088247885a4, user_id=f1462d1f526745aabf69e60f82e8ba8d, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:02.293629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf264563-5c5c-4d71-9606-bc53a3570d7e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:02.485623 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1202/2392] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:02 2022] PUT /identity/v3/projects/e61a5d896efe44d2a41c4088247885a4/users/f1462d1f526745aabf69e60f82e8ba8d/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:02.496512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e69cf-a65f-44bd-a886-56e711426670 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:02.496808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e69cf-a65f-44bd-a886-56e711426670 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:02.497090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e69cf-a65f-44bd-a886-56e711426670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:02.527749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-9e8e69cf-a65f-44bd-a886-56e711426670 None None] MFA Rules not processed for user `f1462d1f526745aabf69e60f82e8ba8d`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:02.528917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9e8e69cf-a65f-44bd-a886-56e711426670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:02.601045 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1191/2393] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:02.605613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf264563-5c5c-4d71-9606-bc53a3570d7e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:02.673165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf264563-5c5c-4d71-9606-bc53a3570d7e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:02.678011 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99cd82-aa85-427f-ad3c-3010956b6366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:02.678762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99cd82-aa85-427f-ad3c-3010956b6366 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:02.679049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99cd82-aa85-427f-ad3c-3010956b6366 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:02.679333 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99cd82-aa85-427f-ad3c-3010956b6366 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:02.680033 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99cd82-aa85-427f-ad3c-3010956b6366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:02.681985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99cd82-aa85-427f-ad3c-3010956b6366 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:02.688053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:02.829908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1203/2394] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:02 2022] POST /identity/v3/projects => generated 369 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:16:02.834008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e69cf-a65f-44bd-a886-56e711426670 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:02.846141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e69cf-a65f-44bd-a886-56e711426670 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:02.897941 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:02.913956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8114a-8322-4ab2-b99e-de646c4634bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:02.914732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8114a-8322-4ab2-b99e-de646c4634bf admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:02.915073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8114a-8322-4ab2-b99e-de646c4634bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:02.915314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8114a-8322-4ab2-b99e-de646c4634bf admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:02.916068 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8114a-8322-4ab2-b99e-de646c4634bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:02.917999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8114a-8322-4ab2-b99e-de646c4634bf admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:02.996437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1192/2395] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:02 2022] POST /identity/v3/users => generated 291 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:03.001223 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99cd82-aa85-427f-ad3c-3010956b6366 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:03.012482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99cd82-aa85-427f-ad3c-3010956b6366 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:03.017436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21469668-163c-4b54-b7e9-0d66b51399c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:03.018185 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21469668-163c-4b54-b7e9-0d66b51399c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:03.018501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21469668-163c-4b54-b7e9-0d66b51399c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:03.018792 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21469668-163c-4b54-b7e9-0d66b51399c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:03.019537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21469668-163c-4b54-b7e9-0d66b51399c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:03.021172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21469668-163c-4b54-b7e9-0d66b51399c4 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:03.050963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1204/2396] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:16:03.051141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8114a-8322-4ab2-b99e-de646c4634bf admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:03.061663 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8114a-8322-4ab2-b99e-de646c4634bf admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:03.066607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510ebca-74b0-4388-8683-2c766872aa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:03.067512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510ebca-74b0-4388-8683-2c766872aa90 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:03.067801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510ebca-74b0-4388-8683-2c766872aa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:03.068079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510ebca-74b0-4388-8683-2c766872aa90 admin admin] PATH_INFO: `/v3/projects/277c5b35298e4e4d905084273ece89f3/users/3335af571025405fad45db2ab27acce8/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:03.084183 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510ebca-74b0-4388-8683-2c766872aa90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277c5b35298e4e4d905084273ece89f3, user_id=3335af571025405fad45db2ab27acce8, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:03.086757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510ebca-74b0-4388-8683-2c766872aa90 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:03.160116 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1193/2397] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:03 2022] PUT /identity/v3/projects/277c5b35298e4e4d905084273ece89f3/users/3335af571025405fad45db2ab27acce8/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:03.161993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21469668-163c-4b54-b7e9-0d66b51399c4 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:03.173791 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21469668-163c-4b54-b7e9-0d66b51399c4 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:03.237542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165fa83-09d6-4a8f-8057-8457f12d42b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:03.238370 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165fa83-09d6-4a8f-8057-8457f12d42b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:03.238667 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165fa83-09d6-4a8f-8057-8457f12d42b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:03.238947 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165fa83-09d6-4a8f-8057-8457f12d42b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:03.239697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165fa83-09d6-4a8f-8057-8457f12d42b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:03.241436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165fa83-09d6-4a8f-8057-8457f12d42b8 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:03.253984 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1205/2398] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:03.258318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510ebca-74b0-4388-8683-2c766872aa90 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:03.270316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510ebca-74b0-4388-8683-2c766872aa90 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:03.275210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e6f37-dae9-4fee-b4e2-7ccd76c11de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:03.276119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e6f37-dae9-4fee-b4e2-7ccd76c11de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:03.276413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e6f37-dae9-4fee-b4e2-7ccd76c11de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:03.276704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e6f37-dae9-4fee-b4e2-7ccd76c11de9 admin admin] PATH_INFO: `/v3/projects/277c5b35298e4e4d905084273ece89f3/users/3335af571025405fad45db2ab27acce8/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:03.278826 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e6f37-dae9-4fee-b4e2-7ccd76c11de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277c5b35298e4e4d905084273ece89f3, user_id=3335af571025405fad45db2ab27acce8, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:03.282959 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e6f37-dae9-4fee-b4e2-7ccd76c11de9 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:03.294514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1194/2399] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:03 2022] PUT /identity/v3/projects/277c5b35298e4e4d905084273ece89f3/users/3335af571025405fad45db2ab27acce8/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:03.307123 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec53ed-611b-469a-ac47-780036fb4f23 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:03.307489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec53ed-611b-469a-ac47-780036fb4f23 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:03.307868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec53ed-611b-469a-ac47-780036fb4f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:03.339976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-18ec53ed-611b-469a-ac47-780036fb4f23 None None] MFA Rules not processed for user `3335af571025405fad45db2ab27acce8`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:03.341523 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-18ec53ed-611b-469a-ac47-780036fb4f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:03.426550 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1206/2400] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:03.437390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e6f37-dae9-4fee-b4e2-7ccd76c11de9 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:03.440091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ee9e6f37-dae9-4fee-b4e2-7ccd76c11de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:03.500166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e6f37-dae9-4fee-b4e2-7ccd76c11de9 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:03.504471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064b8ea-cc55-49e8-8072-d42874b2ed98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:03.505165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064b8ea-cc55-49e8-8072-d42874b2ed98 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:03.505507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064b8ea-cc55-49e8-8072-d42874b2ed98 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:03.505821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064b8ea-cc55-49e8-8072-d42874b2ed98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:03.516588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064b8ea-cc55-49e8-8072-d42874b2ed98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:03.518433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064b8ea-cc55-49e8-8072-d42874b2ed98 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:03.531044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1195/2401] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:03 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:06.992448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec53ed-611b-469a-ac47-780036fb4f23 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:07.056192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:07.060673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec53ed-611b-469a-ac47-780036fb4f23 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:07.065906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:07.067104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:07.067462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:07.067750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] PATH_INFO: `/v3/users/f1462d1f526745aabf69e60f82e8ba8d` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:07.069012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1462d1f526745aabf69e60f82e8ba8d)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:07.071127 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:07.267697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1462d1f526745aabf69e60f82e8ba8d'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:07.268367 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1462d1f526745aabf69e60f82e8ba8d'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:07.275417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1462d1f526745aabf69e60f82e8ba8d'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:07.291494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1462d1f526745aabf69e60f82e8ba8d'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:07.401818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1462d1f526745aabf69e60f82e8ba8d'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:07.413290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1207/2402] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:06 2022] DELETE /identity/v3/users/f1462d1f526745aabf69e60f82e8ba8d => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:07.417904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d4f85-4c4e-41c2-a4d5-a67a38586730 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:07.418390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d4f85-4c4e-41c2-a4d5-a67a38586730 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:07.418491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d4f85-4c4e-41c2-a4d5-a67a38586730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:07.437980 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e40d4f85-4c4e-41c2-a4d5-a67a38586730 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:07.439435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e40d4f85-4c4e-41c2-a4d5-a67a38586730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:07.518000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1196/2403] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:07.529022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:07.531312 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:07.598544 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e043f62-f9a8-4583-a475-d9ab2763a364 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:07.603991 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34434ed1-c6a5-4628-94ce-755744e7793d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:07.604777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34434ed1-c6a5-4628-94ce-755744e7793d service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:07.605094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34434ed1-c6a5-4628-94ce-755744e7793d service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:07.605377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34434ed1-c6a5-4628-94ce-755744e7793d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:07.616691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34434ed1-c6a5-4628-94ce-755744e7793d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:07.618151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34434ed1-c6a5-4628-94ce-755744e7793d service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:07.635433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1208/2404] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:08.302190 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d4f85-4c4e-41c2-a4d5-a67a38586730 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:08.303751 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e40d4f85-4c4e-41c2-a4d5-a67a38586730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:08.316909 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d4f85-4c4e-41c2-a4d5-a67a38586730 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:08.322131 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fee908-de2f-42ba-b54f-a30016ce1eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:08.323039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fee908-de2f-42ba-b54f-a30016ce1eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:08.323319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fee908-de2f-42ba-b54f-a30016ce1eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:08.323594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fee908-de2f-42ba-b54f-a30016ce1eb3 admin admin] PATH_INFO: `/v3/projects/e61a5d896efe44d2a41c4088247885a4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:08.324943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fee908-de2f-42ba-b54f-a30016ce1eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e61a5d896efe44d2a41c4088247885a4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:08.327198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fee908-de2f-42ba-b54f-a30016ce1eb3 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:08.446106 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:08.457466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-b9fee908-de2f-42ba-b54f-a30016ce1eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e61a5d896efe44d2a41c4088247885a4', 'request_id': 'req-b9fee908-de2f-42ba-b54f-a30016ce1eb3'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:08.521442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-b9fee908-de2f-42ba-b54f-a30016ce1eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e61a5d896efe44d2a41c4088247885a4', 'request_id': 'req-b9fee908-de2f-42ba-b54f-a30016ce1eb3'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:08.527538 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-b9fee908-de2f-42ba-b54f-a30016ce1eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e61a5d896efe44d2a41c4088247885a4', 'request_id': 'req-b9fee908-de2f-42ba-b54f-a30016ce1eb3'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:08.540483 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-b9fee908-de2f-42ba-b54f-a30016ce1eb3 admin admin] The token cache is being invalidate because project e61a5d896efe44d2a41c4088247885a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:08.540782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-b9fee908-de2f-42ba-b54f-a30016ce1eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:08.541504 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-b9fee908-de2f-42ba-b54f-a30016ce1eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:08.543199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1197/2405] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:08 2022] DELETE /identity/v3/projects/e61a5d896efe44d2a41c4088247885a4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:08.547658 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34434ed1-c6a5-4628-94ce-755744e7793d service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:08.558600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34434ed1-c6a5-4628-94ce-755744e7793d service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:08.627424 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a33d1b-086f-4425-bae6-ad3a1072a18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:08.627976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a33d1b-086f-4425-bae6-ad3a1072a18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:08.628482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a33d1b-086f-4425-bae6-ad3a1072a18a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:08.628844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a33d1b-086f-4425-bae6-ad3a1072a18a admin admin] PATH_INFO: `/v3/users/3335af571025405fad45db2ab27acce8` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:08.630256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a33d1b-086f-4425-bae6-ad3a1072a18a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3335af571025405fad45db2ab27acce8)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:08.632295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a33d1b-086f-4425-bae6-ad3a1072a18a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:08.656150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:08.744430 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222b76e-3eeb-4738-8703-3256c328f329 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:08.744752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222b76e-3eeb-4738-8703-3256c328f329 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:08.745039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222b76e-3eeb-4738-8703-3256c328f329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:08.775987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-0222b76e-3eeb-4738-8703-3256c328f329 None None] MFA Rules not processed for user `592de777b94148b791e18bf898386f34`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:08.777331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0222b76e-3eeb-4738-8703-3256c328f329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:08.838973 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1198/2406] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:16:08.860938 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73a33d1b-086f-4425-bae6-ad3a1072a18a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3335af571025405fad45db2ab27acce8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:08.862262 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73a33d1b-086f-4425-bae6-ad3a1072a18a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3335af571025405fad45db2ab27acce8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:08.869859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73a33d1b-086f-4425-bae6-ad3a1072a18a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3335af571025405fad45db2ab27acce8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:08.886162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73a33d1b-086f-4425-bae6-ad3a1072a18a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3335af571025405fad45db2ab27acce8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:08.963725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73a33d1b-086f-4425-bae6-ad3a1072a18a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3335af571025405fad45db2ab27acce8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:08.970580 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1209/2407] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:08 2022] DELETE /identity/v3/users/3335af571025405fad45db2ab27acce8 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:09.581245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222b76e-3eeb-4738-8703-3256c328f329 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:09.582780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0222b76e-3eeb-4738-8703-3256c328f329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:09.597404 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222b76e-3eeb-4738-8703-3256c328f329 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:09.664213 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d46bcd-d8d5-4906-a047-186131a100ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:09.665479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d46bcd-d8d5-4906-a047-186131a100ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:09.665780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d46bcd-d8d5-4906-a047-186131a100ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:09.666066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d46bcd-d8d5-4906-a047-186131a100ab admin admin] PATH_INFO: `/v3/projects/277c5b35298e4e4d905084273ece89f3` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:09.667390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d46bcd-d8d5-4906-a047-186131a100ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=277c5b35298e4e4d905084273ece89f3)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:09.669401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d46bcd-d8d5-4906-a047-186131a100ab admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:09.786618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-24d46bcd-d8d5-4906-a047-186131a100ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277c5b35298e4e4d905084273ece89f3', 'request_id': 'req-24d46bcd-d8d5-4906-a047-186131a100ab'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:09.866708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-24d46bcd-d8d5-4906-a047-186131a100ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277c5b35298e4e4d905084273ece89f3', 'request_id': 'req-24d46bcd-d8d5-4906-a047-186131a100ab'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:09.872777 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:09.874167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-24d46bcd-d8d5-4906-a047-186131a100ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277c5b35298e4e4d905084273ece89f3', 'request_id': 'req-24d46bcd-d8d5-4906-a047-186131a100ab'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:09.887665 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-24d46bcd-d8d5-4906-a047-186131a100ab admin admin] The token cache is being invalidate because project 277c5b35298e4e4d905084273ece89f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:09.888005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-24d46bcd-d8d5-4906-a047-186131a100ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:09.888648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-24d46bcd-d8d5-4906-a047-186131a100ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:09.890231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1199/2408] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:09 2022] DELETE /identity/v3/projects/277c5b35298e4e4d905084273ece89f3 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:09.895618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd93742-0a94-41da-a060-d2e8aa146162 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:09.896153 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd93742-0a94-41da-a060-d2e8aa146162 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:09.896314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd93742-0a94-41da-a060-d2e8aa146162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:09.918692 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-7dd93742-0a94-41da-a060-d2e8aa146162 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:09.919963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7dd93742-0a94-41da-a060-d2e8aa146162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:09.983512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1210/2409] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:09.992094 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f8b14-df6e-4254-ae11-7a0137746b10 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:09.992386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f8b14-df6e-4254-ae11-7a0137746b10 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:09.992669 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f8b14-df6e-4254-ae11-7a0137746b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:10.013216 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-890f8b14-df6e-4254-ae11-7a0137746b10 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:10.014297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-890f8b14-df6e-4254-ae11-7a0137746b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:10.022811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1200/2410] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:10.027189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd93742-0a94-41da-a060-d2e8aa146162 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:10.038441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd93742-0a94-41da-a060-d2e8aa146162 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:10.043159 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886be646-a178-45b7-91a2-1cbf781b8330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:10.043833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886be646-a178-45b7-91a2-1cbf781b8330 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:10.044121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886be646-a178-45b7-91a2-1cbf781b8330 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:10.044402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886be646-a178-45b7-91a2-1cbf781b8330 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:10.045132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886be646-a178-45b7-91a2-1cbf781b8330 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:16:10.045446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886be646-a178-45b7-91a2-1cbf781b8330 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:10.047687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886be646-a178-45b7-91a2-1cbf781b8330 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:10.060433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1211/2411] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:16:10.064957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f8b14-df6e-4254-ae11-7a0137746b10 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:10.074463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f8b14-df6e-4254-ae11-7a0137746b10 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:10.079589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b454286-5ba1-49ca-a51d-130ba5fe4bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:10.080179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454286-5ba1-49ca-a51d-130ba5fe4bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:10.080686 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454286-5ba1-49ca-a51d-130ba5fe4bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:10.080833 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454286-5ba1-49ca-a51d-130ba5fe4bf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:10.081486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b454286-5ba1-49ca-a51d-130ba5fe4bf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:10.083419 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b454286-5ba1-49ca-a51d-130ba5fe4bf5 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:10.169471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1201/2412] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:10 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:10.173977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886be646-a178-45b7-91a2-1cbf781b8330 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:10.184364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886be646-a178-45b7-91a2-1cbf781b8330 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:10.244182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e3891-a2d0-4e6b-b51c-54590862ea8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:10.244864 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e3891-a2d0-4e6b-b51c-54590862ea8d admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:10.245156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e3891-a2d0-4e6b-b51c-54590862ea8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:10.245696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e3891-a2d0-4e6b-b51c-54590862ea8d admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:10.246436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e3891-a2d0-4e6b-b51c-54590862ea8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:10.248378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e3891-a2d0-4e6b-b51c-54590862ea8d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:10.337203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1212/2413] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:10 2022] POST /identity/v3/users => generated 294 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:10.341371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b454286-5ba1-49ca-a51d-130ba5fe4bf5 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:10.352740 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b454286-5ba1-49ca-a51d-130ba5fe4bf5 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:10.357413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b94760-0948-40ad-9cd2-7e809d1d1113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:10.358109 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b94760-0948-40ad-9cd2-7e809d1d1113 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:10.358426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b94760-0948-40ad-9cd2-7e809d1d1113 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:10.358708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b94760-0948-40ad-9cd2-7e809d1d1113 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:10.359550 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b94760-0948-40ad-9cd2-7e809d1d1113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:10.361311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b94760-0948-40ad-9cd2-7e809d1d1113 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:10.372337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1202/2414] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:10.376955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e3891-a2d0-4e6b-b51c-54590862ea8d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:10.387535 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e3891-a2d0-4e6b-b51c-54590862ea8d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:10.393455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2995dd5-ea09-47e3-9749-92dee4631643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:10.394387 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2995dd5-ea09-47e3-9749-92dee4631643 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:10.394677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2995dd5-ea09-47e3-9749-92dee4631643 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:10.394956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2995dd5-ea09-47e3-9749-92dee4631643 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:10.395698 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2995dd5-ea09-47e3-9749-92dee4631643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:10.397358 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2995dd5-ea09-47e3-9749-92dee4631643 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:10.409348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1213/2415] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:16:10.413659 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b94760-0948-40ad-9cd2-7e809d1d1113 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:10.424369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b94760-0948-40ad-9cd2-7e809d1d1113 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:10.429220 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8903ac-b0cc-4989-bd61-888d5e02f54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:10.430113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8903ac-b0cc-4989-bd61-888d5e02f54b admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:10.430533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8903ac-b0cc-4989-bd61-888d5e02f54b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:10.430986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8903ac-b0cc-4989-bd61-888d5e02f54b admin admin] PATH_INFO: `/v3/projects/ee8fa9d234ff418aa2310c85c19ccaee/users/122110112fac49a29dc8755aa3b37a43/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:10.447526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8903ac-b0cc-4989-bd61-888d5e02f54b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8fa9d234ff418aa2310c85c19ccaee, user_id=122110112fac49a29dc8755aa3b37a43, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:10.450181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8903ac-b0cc-4989-bd61-888d5e02f54b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:10.522644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1203/2416] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:10 2022] PUT /identity/v3/projects/ee8fa9d234ff418aa2310c85c19ccaee/users/122110112fac49a29dc8755aa3b37a43/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:10.535194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c8376-0805-4c6c-80d6-58de6c690686 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:10.535493 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c8376-0805-4c6c-80d6-58de6c690686 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:10.535781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c8376-0805-4c6c-80d6-58de6c690686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:10.544138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:10.566251 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-745c8376-0805-4c6c-80d6-58de6c690686 None None] MFA Rules not processed for user `122110112fac49a29dc8755aa3b37a43`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:10.568215 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-745c8376-0805-4c6c-80d6-58de6c690686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:10.637866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1214/2417] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:10.642984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8903ac-b0cc-4989-bd61-888d5e02f54b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:10.703596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c8376-0805-4c6c-80d6-58de6c690686 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:10.705363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-745c8376-0805-4c6c-80d6-58de6c690686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:10.714608 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8903ac-b0cc-4989-bd61-888d5e02f54b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:10.720530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45e166-e288-45b4-8a42-f05c9cd5449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:10.721221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45e166-e288-45b4-8a42-f05c9cd5449a admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:10.721522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45e166-e288-45b4-8a42-f05c9cd5449a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:10.721807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45e166-e288-45b4-8a42-f05c9cd5449a admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:10.722555 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45e166-e288-45b4-8a42-f05c9cd5449a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:10.724494 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45e166-e288-45b4-8a42-f05c9cd5449a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:10.780745 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c8376-0805-4c6c-80d6-58de6c690686 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:10.788958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946153d-3e2d-4506-8ed8-c7d54d717161 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:10.789643 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946153d-3e2d-4506-8ed8-c7d54d717161 service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:10.789932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946153d-3e2d-4506-8ed8-c7d54d717161 service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:10.790219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946153d-3e2d-4506-8ed8-c7d54d717161 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:10.792102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b946153d-3e2d-4506-8ed8-c7d54d717161 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:10.805391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946153d-3e2d-4506-8ed8-c7d54d717161 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:10.807065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946153d-3e2d-4506-8ed8-c7d54d717161 service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:10.818317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1215/2418] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:16:10.849387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1204/2419] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:10 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:10.882239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946153d-3e2d-4506-8ed8-c7d54d717161 service placement] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:10.893609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946153d-3e2d-4506-8ed8-c7d54d717161 service placement] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:10.955877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45f484-91b5-46a1-a9e5-e26baa6eb781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:10.956618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45f484-91b5-46a1-a9e5-e26baa6eb781 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:10.956832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45f484-91b5-46a1-a9e5-e26baa6eb781 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:10.957133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45f484-91b5-46a1-a9e5-e26baa6eb781 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:10.957838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45f484-91b5-46a1-a9e5-e26baa6eb781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:10.959801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45f484-91b5-46a1-a9e5-e26baa6eb781 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:11.078967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1216/2420] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:10 2022] POST /identity/v3/users => generated 300 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:16:11.083130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45e166-e288-45b4-8a42-f05c9cd5449a admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:11.094642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45e166-e288-45b4-8a42-f05c9cd5449a admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:11.099257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257b4ab-f593-4658-b47d-a2110c70050c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:11.103698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257b4ab-f593-4658-b47d-a2110c70050c admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:11.103698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257b4ab-f593-4658-b47d-a2110c70050c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:11.103698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257b4ab-f593-4658-b47d-a2110c70050c admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:11.103698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257b4ab-f593-4658-b47d-a2110c70050c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:11.103698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257b4ab-f593-4658-b47d-a2110c70050c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:11.114248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1205/2421] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:11.118568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45f484-91b5-46a1-a9e5-e26baa6eb781 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:11.129895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45f484-91b5-46a1-a9e5-e26baa6eb781 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:11.134920 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2145c2-f34f-404e-b271-43a05c24e57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:11.135827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2145c2-f34f-404e-b271-43a05c24e57d admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:11.136115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2145c2-f34f-404e-b271-43a05c24e57d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:11.136395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2145c2-f34f-404e-b271-43a05c24e57d admin admin] PATH_INFO: `/v3/projects/ae95fc413e994093b322bcfa6407182f/users/5614d75703aa44d8a31f704989aa368b/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:11.149057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:11.153113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2145c2-f34f-404e-b271-43a05c24e57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae95fc413e994093b322bcfa6407182f, user_id=5614d75703aa44d8a31f704989aa368b, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:11.155917 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2145c2-f34f-404e-b271-43a05c24e57d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:11.236828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1217/2422] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:11 2022] PUT /identity/v3/projects/ae95fc413e994093b322bcfa6407182f/users/5614d75703aa44d8a31f704989aa368b/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:11.241297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257b4ab-f593-4658-b47d-a2110c70050c admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:11.252546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257b4ab-f593-4658-b47d-a2110c70050c admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:11.313940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fcf5b-751a-4ee7-a421-2783f6a02fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:11.314677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fcf5b-751a-4ee7-a421-2783f6a02fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:11.314964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fcf5b-751a-4ee7-a421-2783f6a02fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:11.315237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fcf5b-751a-4ee7-a421-2783f6a02fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:11.315962 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fcf5b-751a-4ee7-a421-2783f6a02fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:11.318526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fcf5b-751a-4ee7-a421-2783f6a02fc1 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:11.322425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:11.333190 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1206/2423] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:11.337273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2145c2-f34f-404e-b271-43a05c24e57d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:11.347989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2145c2-f34f-404e-b271-43a05c24e57d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:11.353027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a996608-fa06-4cd6-bf53-cf50b58870c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:11.354045 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a996608-fa06-4cd6-bf53-cf50b58870c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:11.354357 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a996608-fa06-4cd6-bf53-cf50b58870c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:11.354675 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a996608-fa06-4cd6-bf53-cf50b58870c4 admin admin] PATH_INFO: `/v3/projects/ae95fc413e994093b322bcfa6407182f/users/5614d75703aa44d8a31f704989aa368b/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:11.356641 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a996608-fa06-4cd6-bf53-cf50b58870c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae95fc413e994093b322bcfa6407182f, user_id=5614d75703aa44d8a31f704989aa368b, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:11.359384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a996608-fa06-4cd6-bf53-cf50b58870c4 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:11.370578 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1218/2424] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:11 2022] PUT /identity/v3/projects/ae95fc413e994093b322bcfa6407182f/users/5614d75703aa44d8a31f704989aa368b/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:11.386874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746d77d-35c9-43f7-a387-b53cfd0c7eb4 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:11.387169 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746d77d-35c9-43f7-a387-b53cfd0c7eb4 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:11.387456 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746d77d-35c9-43f7-a387-b53cfd0c7eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:11.418737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b746d77d-35c9-43f7-a387-b53cfd0c7eb4 None None] MFA Rules not processed for user `5614d75703aa44d8a31f704989aa368b`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:11.419890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b746d77d-35c9-43f7-a387-b53cfd0c7eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:11.500911 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1207/2425] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:11.512300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a996608-fa06-4cd6-bf53-cf50b58870c4 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:11.514051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9a996608-fa06-4cd6-bf53-cf50b58870c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:11.573682 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a996608-fa06-4cd6-bf53-cf50b58870c4 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:11.577392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f1731-ca39-4384-a439-07eeb6da4af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:11.578635 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f1731-ca39-4384-a439-07eeb6da4af4 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:11.578925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f1731-ca39-4384-a439-07eeb6da4af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:11.579210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f1731-ca39-4384-a439-07eeb6da4af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:11.588531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f1731-ca39-4384-a439-07eeb6da4af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:11.589975 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f1731-ca39-4384-a439-07eeb6da4af4 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:11.599884 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1219/2426] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:11.678723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746d77d-35c9-43f7-a387-b53cfd0c7eb4 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:11.743836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746d77d-35c9-43f7-a387-b53cfd0c7eb4 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:11.748857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:11.750352 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:11.750668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:11.750961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] PATH_INFO: `/v3/users/122110112fac49a29dc8755aa3b37a43` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:11.752275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=122110112fac49a29dc8755aa3b37a43)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:11.754229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:11.967096 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '122110112fac49a29dc8755aa3b37a43'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:11.967940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '122110112fac49a29dc8755aa3b37a43'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:11.968918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '122110112fac49a29dc8755aa3b37a43'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:12.055224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '122110112fac49a29dc8755aa3b37a43'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:12.069121 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '122110112fac49a29dc8755aa3b37a43'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:12.081838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1208/2427] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:11 2022] DELETE /identity/v3/users/122110112fac49a29dc8755aa3b37a43 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:12.086576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096ef60-1c83-4034-8581-8113e745673c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:12.087043 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096ef60-1c83-4034-8581-8113e745673c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:12.087218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096ef60-1c83-4034-8581-8113e745673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:12.108078 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-b096ef60-1c83-4034-8581-8113e745673c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:12.110150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b096ef60-1c83-4034-8581-8113e745673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:12.184841 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1220/2428] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:12.194016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:12.195528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:12.200948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:12.259620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef409c0-10f2-4c71-979a-c59023e50522 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:12.263338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab1634-5275-4917-9bae-cf02a9b7edeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:12.264269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab1634-5275-4917-9bae-cf02a9b7edeb service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:12.264732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab1634-5275-4917-9bae-cf02a9b7edeb service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:12.266067 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab1634-5275-4917-9bae-cf02a9b7edeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:12.276128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab1634-5275-4917-9bae-cf02a9b7edeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:12.277581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab1634-5275-4917-9bae-cf02a9b7edeb service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:12.288198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1209/2429] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:12.720068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096ef60-1c83-4034-8581-8113e745673c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:12.721499 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b096ef60-1c83-4034-8581-8113e745673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:12.734861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096ef60-1c83-4034-8581-8113e745673c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:12.739751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:12.740645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:12.740930 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:12.741210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] PATH_INFO: `/v3/projects/ee8fa9d234ff418aa2310c85c19ccaee` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:12.742595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee8fa9d234ff418aa2310c85c19ccaee)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:12.744574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:12.884980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8fa9d234ff418aa2310c85c19ccaee', 'request_id': 'req-f7ffee8b-8594-4309-ab74-eada68ec6799'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:12.891753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8fa9d234ff418aa2310c85c19ccaee', 'request_id': 'req-f7ffee8b-8594-4309-ab74-eada68ec6799'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:12.898013 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8fa9d234ff418aa2310c85c19ccaee', 'request_id': 'req-f7ffee8b-8594-4309-ab74-eada68ec6799'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:13.001023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] The token cache is being invalidate because project ee8fa9d234ff418aa2310c85c19ccaee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:13.001023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:13.002227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:13.004569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1221/2430] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:12 2022] DELETE /identity/v3/projects/ee8fa9d234ff418aa2310c85c19ccaee => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:13.009939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab1634-5275-4917-9bae-cf02a9b7edeb service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:13.020641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab1634-5275-4917-9bae-cf02a9b7edeb service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:13.094632 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24809476-7884-44f5-a4e6-e0c6f9cf9a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:13.095536 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24809476-7884-44f5-a4e6-e0c6f9cf9a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:13.095823 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24809476-7884-44f5-a4e6-e0c6f9cf9a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:13.096102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24809476-7884-44f5-a4e6-e0c6f9cf9a79 admin admin] PATH_INFO: `/v3/users/5614d75703aa44d8a31f704989aa368b` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:13.097343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24809476-7884-44f5-a4e6-e0c6f9cf9a79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5614d75703aa44d8a31f704989aa368b)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:13.099340 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24809476-7884-44f5-a4e6-e0c6f9cf9a79 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:13.293199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-24809476-7884-44f5-a4e6-e0c6f9cf9a79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5614d75703aa44d8a31f704989aa368b'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:13.293863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-24809476-7884-44f5-a4e6-e0c6f9cf9a79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5614d75703aa44d8a31f704989aa368b'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:13.294510 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-24809476-7884-44f5-a4e6-e0c6f9cf9a79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5614d75703aa44d8a31f704989aa368b'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:13.418549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-24809476-7884-44f5-a4e6-e0c6f9cf9a79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5614d75703aa44d8a31f704989aa368b'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:13.434885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-24809476-7884-44f5-a4e6-e0c6f9cf9a79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5614d75703aa44d8a31f704989aa368b'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:13.449904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1210/2431] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:13 2022] DELETE /identity/v3/users/5614d75703aa44d8a31f704989aa368b => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:13.963356 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:13.964889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:13.975200 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:13.980912 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffee8b-8594-4309-ab74-eada68ec6799 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:14.042263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4e4dc-602a-444a-be7e-091bfd6f505f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:14.043204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4e4dc-602a-444a-be7e-091bfd6f505f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:14.043498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4e4dc-602a-444a-be7e-091bfd6f505f admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:14.043811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4e4dc-602a-444a-be7e-091bfd6f505f admin admin] PATH_INFO: `/v3/projects/ae95fc413e994093b322bcfa6407182f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:14.045032 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4e4dc-602a-444a-be7e-091bfd6f505f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae95fc413e994093b322bcfa6407182f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:14.047063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4e4dc-602a-444a-be7e-091bfd6f505f admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:14.200009 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-74b4e4dc-602a-444a-be7e-091bfd6f505f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae95fc413e994093b322bcfa6407182f', 'request_id': 'req-74b4e4dc-602a-444a-be7e-091bfd6f505f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:14.207999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-74b4e4dc-602a-444a-be7e-091bfd6f505f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae95fc413e994093b322bcfa6407182f', 'request_id': 'req-74b4e4dc-602a-444a-be7e-091bfd6f505f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:14.214399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-74b4e4dc-602a-444a-be7e-091bfd6f505f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae95fc413e994093b322bcfa6407182f', 'request_id': 'req-74b4e4dc-602a-444a-be7e-091bfd6f505f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:14.316221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-74b4e4dc-602a-444a-be7e-091bfd6f505f admin admin] The token cache is being invalidate because project ae95fc413e994093b322bcfa6407182f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:14.316559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-74b4e4dc-602a-444a-be7e-091bfd6f505f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:14.317239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-74b4e4dc-602a-444a-be7e-091bfd6f505f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:14.318865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1222/2432] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:13 2022] DELETE /identity/v3/projects/ae95fc413e994093b322bcfa6407182f => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:14.323858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d28a7f-8117-4918-b100-1834db52a5cc None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:14.324343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d28a7f-8117-4918-b100-1834db52a5cc None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:14.324463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d28a7f-8117-4918-b100-1834db52a5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:14.345668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-77d28a7f-8117-4918-b100-1834db52a5cc None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:14.346970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-77d28a7f-8117-4918-b100-1834db52a5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:14.372955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:14.410644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1211/2433] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:14.418499 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ddf73-d716-4378-bfaa-8c1fc006fcc5 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:14.423800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ddf73-d716-4378-bfaa-8c1fc006fcc5 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:14.423800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ddf73-d716-4378-bfaa-8c1fc006fcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:14.440772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-a00ddf73-d716-4378-bfaa-8c1fc006fcc5 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:14.442010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a00ddf73-d716-4378-bfaa-8c1fc006fcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:14.450713 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1223/2434] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:14.455721 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d28a7f-8117-4918-b100-1834db52a5cc None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:14.465661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d28a7f-8117-4918-b100-1834db52a5cc None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:14.471970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85bdd6-9cf4-44e2-8d72-50e05920793c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:14.472642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85bdd6-9cf4-44e2-8d72-50e05920793c admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:14.472932 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85bdd6-9cf4-44e2-8d72-50e05920793c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:14.473215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85bdd6-9cf4-44e2-8d72-50e05920793c admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:14.474023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85bdd6-9cf4-44e2-8d72-50e05920793c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:16:14.474263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85bdd6-9cf4-44e2-8d72-50e05920793c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:14.475977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85bdd6-9cf4-44e2-8d72-50e05920793c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:14.488946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1212/2435] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:16:14.493702 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ddf73-d716-4378-bfaa-8c1fc006fcc5 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:14.505361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ddf73-d716-4378-bfaa-8c1fc006fcc5 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:14.510398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00ead1-fa42-4ee8-8df2-5fbf8a23a63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:14.511103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00ead1-fa42-4ee8-8df2-5fbf8a23a63b admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:14.511388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00ead1-fa42-4ee8-8df2-5fbf8a23a63b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:14.511664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00ead1-fa42-4ee8-8df2-5fbf8a23a63b admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:14.512349 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00ead1-fa42-4ee8-8df2-5fbf8a23a63b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:14.514253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00ead1-fa42-4ee8-8df2-5fbf8a23a63b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:14.644740 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1224/2436] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:14 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:14.648987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85bdd6-9cf4-44e2-8d72-50e05920793c admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:14.659736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85bdd6-9cf4-44e2-8d72-50e05920793c admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:14.716887 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1c8ce-ac1e-484e-b3e8-55e1838ea815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:14.717563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c8ce-ac1e-484e-b3e8-55e1838ea815 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:14.717881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c8ce-ac1e-484e-b3e8-55e1838ea815 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:14.718162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c8ce-ac1e-484e-b3e8-55e1838ea815 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:14.719284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1c8ce-ac1e-484e-b3e8-55e1838ea815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:14.721210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1c8ce-ac1e-484e-b3e8-55e1838ea815 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:14.821408 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1213/2437] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:14 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:14.825701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00ead1-fa42-4ee8-8df2-5fbf8a23a63b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:14.836372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00ead1-fa42-4ee8-8df2-5fbf8a23a63b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:14.841146 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ae6da-dfc8-4877-9aed-6786b4d3e0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:14.841818 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ae6da-dfc8-4877-9aed-6786b4d3e0fe admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:14.842103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ae6da-dfc8-4877-9aed-6786b4d3e0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:14.842410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ae6da-dfc8-4877-9aed-6786b4d3e0fe admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:14.843135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ae6da-dfc8-4877-9aed-6786b4d3e0fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:14.844746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ae6da-dfc8-4877-9aed-6786b4d3e0fe admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:14.855802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1225/2438] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:16:14.865948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1c8ce-ac1e-484e-b3e8-55e1838ea815 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:14.877259 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1c8ce-ac1e-484e-b3e8-55e1838ea815 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:14.882952 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733e1e9-4fdf-4b22-8df7-290cdd63758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:14.883672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733e1e9-4fdf-4b22-8df7-290cdd63758c admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:14.883957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733e1e9-4fdf-4b22-8df7-290cdd63758c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:14.884233 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733e1e9-4fdf-4b22-8df7-290cdd63758c admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:14.884967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733e1e9-4fdf-4b22-8df7-290cdd63758c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:14.886748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733e1e9-4fdf-4b22-8df7-290cdd63758c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:14.899027 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1214/2439] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:16:14.903298 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ae6da-dfc8-4877-9aed-6786b4d3e0fe admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:14.914534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ae6da-dfc8-4877-9aed-6786b4d3e0fe admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:14.919305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96557062-6da3-43f3-a535-7dc4f19fe6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:14.920191 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96557062-6da3-43f3-a535-7dc4f19fe6c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:14.920477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96557062-6da3-43f3-a535-7dc4f19fe6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:14.920753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96557062-6da3-43f3-a535-7dc4f19fe6c0 admin admin] PATH_INFO: `/v3/projects/9fb43314dce84f1c97fc18f89149c9b4/users/c0ced36a80e744d18022fa1c34fdf583/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:14.937831 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96557062-6da3-43f3-a535-7dc4f19fe6c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb43314dce84f1c97fc18f89149c9b4, user_id=c0ced36a80e744d18022fa1c34fdf583, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:14.940773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96557062-6da3-43f3-a535-7dc4f19fe6c0 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:15.055576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1226/2440] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:14 2022] PUT /identity/v3/projects/9fb43314dce84f1c97fc18f89149c9b4/users/c0ced36a80e744d18022fa1c34fdf583/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:15.124188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cb25d-ddec-426b-886c-be04418bb6ae None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:15.124512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cb25d-ddec-426b-886c-be04418bb6ae None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:15.125305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cb25d-ddec-426b-886c-be04418bb6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:15.168620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-4b2cb25d-ddec-426b-886c-be04418bb6ae None None] MFA Rules not processed for user `c0ced36a80e744d18022fa1c34fdf583`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:15.169903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4b2cb25d-ddec-426b-886c-be04418bb6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:15.264140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1215/2441] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:15.273950 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96557062-6da3-43f3-a535-7dc4f19fe6c0 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:15.307812 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:15.340933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96557062-6da3-43f3-a535-7dc4f19fe6c0 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:15.347085 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dfbca-f401-495a-a8c4-1d79221783db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:15.347762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dfbca-f401-495a-a8c4-1d79221783db admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:15.348050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dfbca-f401-495a-a8c4-1d79221783db admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:15.348378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dfbca-f401-495a-a8c4-1d79221783db admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:15.349025 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dfbca-f401-495a-a8c4-1d79221783db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:15.351047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dfbca-f401-495a-a8c4-1d79221783db admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:15.497033 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1227/2442] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:15 2022] POST /identity/v3/projects => generated 379 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:16:15.502204 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cb25d-ddec-426b-886c-be04418bb6ae None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:15.513711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cb25d-ddec-426b-886c-be04418bb6ae None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:15.535928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:15.593285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ea340-71c3-43ef-bea1-fd8f6f2ec8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:15.593971 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ea340-71c3-43ef-bea1-fd8f6f2ec8e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:15.594263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ea340-71c3-43ef-bea1-fd8f6f2ec8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:15.594693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ea340-71c3-43ef-bea1-fd8f6f2ec8e2 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:15.595400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ea340-71c3-43ef-bea1-fd8f6f2ec8e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:15.597324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ea340-71c3-43ef-bea1-fd8f6f2ec8e2 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:15.662204 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1216/2443] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:15 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:15.665985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dfbca-f401-495a-a8c4-1d79221783db admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:15.677814 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dfbca-f401-495a-a8c4-1d79221783db admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:15.682903 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2605f-028c-4070-981d-5d0ec469c4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:15.683583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2605f-028c-4070-981d-5d0ec469c4eb admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:15.683870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2605f-028c-4070-981d-5d0ec469c4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:15.684154 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2605f-028c-4070-981d-5d0ec469c4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:15.684923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2605f-028c-4070-981d-5d0ec469c4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:15.686668 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2605f-028c-4070-981d-5d0ec469c4eb admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:15.699343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1228/2444] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:15.703287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ea340-71c3-43ef-bea1-fd8f6f2ec8e2 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:15.713832 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ea340-71c3-43ef-bea1-fd8f6f2ec8e2 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:15.718630 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e6278-0e11-49ea-ac21-47267142206b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:15.719613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e6278-0e11-49ea-ac21-47267142206b admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:15.719912 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e6278-0e11-49ea-ac21-47267142206b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:15.720194 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e6278-0e11-49ea-ac21-47267142206b admin admin] PATH_INFO: `/v3/projects/75da664260b1445b82f2ccd046c9023e/users/2ffb2b9ef360438481973f6a9155b8a8/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:15.736423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e6278-0e11-49ea-ac21-47267142206b admin admin] RBAC: Authorizing `identity:create_grant(project_id=75da664260b1445b82f2ccd046c9023e, user_id=2ffb2b9ef360438481973f6a9155b8a8, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:15.739189 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e6278-0e11-49ea-ac21-47267142206b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:15.834420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1217/2445] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:15 2022] PUT /identity/v3/projects/75da664260b1445b82f2ccd046c9023e/users/2ffb2b9ef360438481973f6a9155b8a8/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:15.838221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2605f-028c-4070-981d-5d0ec469c4eb admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:15.849945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2605f-028c-4070-981d-5d0ec469c4eb admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:15.911866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a9a04-5bfa-4e69-9195-440df0bf8df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:15.912557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a9a04-5bfa-4e69-9195-440df0bf8df8 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:15.912880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a9a04-5bfa-4e69-9195-440df0bf8df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:15.913166 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a9a04-5bfa-4e69-9195-440df0bf8df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:15.913909 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a9a04-5bfa-4e69-9195-440df0bf8df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:15.915541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a9a04-5bfa-4e69-9195-440df0bf8df8 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:15.926970 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1229/2446] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:15.931168 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e6278-0e11-49ea-ac21-47267142206b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:15.942001 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e6278-0e11-49ea-ac21-47267142206b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:15.947044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ae3be-130c-4495-84fc-81cc9032cff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:15.947971 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ae3be-130c-4495-84fc-81cc9032cff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:15.948218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ae3be-130c-4495-84fc-81cc9032cff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:15.948493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ae3be-130c-4495-84fc-81cc9032cff4 admin admin] PATH_INFO: `/v3/projects/75da664260b1445b82f2ccd046c9023e/users/2ffb2b9ef360438481973f6a9155b8a8/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:15.950889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ae3be-130c-4495-84fc-81cc9032cff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75da664260b1445b82f2ccd046c9023e, user_id=2ffb2b9ef360438481973f6a9155b8a8, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:15.954300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ae3be-130c-4495-84fc-81cc9032cff4 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:15.965057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1218/2447] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:15 2022] PUT /identity/v3/projects/75da664260b1445b82f2ccd046c9023e/users/2ffb2b9ef360438481973f6a9155b8a8/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:15.976068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22106c90-f883-4960-b871-947a22ebfcce None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:15.976365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22106c90-f883-4960-b871-947a22ebfcce None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:15.976657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22106c90-f883-4960-b871-947a22ebfcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:16.005654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-22106c90-f883-4960-b871-947a22ebfcce None None] MFA Rules not processed for user `2ffb2b9ef360438481973f6a9155b8a8`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:16.006827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-22106c90-f883-4960-b871-947a22ebfcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:16.085398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1230/2448] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:16.098786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ae3be-130c-4495-84fc-81cc9032cff4 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:16.100551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8c9ae3be-130c-4495-84fc-81cc9032cff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:16.166491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ae3be-130c-4495-84fc-81cc9032cff4 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:16.170094 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2304b-aed3-4912-98d4-e9eea836569a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:16.170812 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2304b-aed3-4912-98d4-e9eea836569a service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:16.171071 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2304b-aed3-4912-98d4-e9eea836569a service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:16.171351 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2304b-aed3-4912-98d4-e9eea836569a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:16.182548 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2304b-aed3-4912-98d4-e9eea836569a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:16.184051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2304b-aed3-4912-98d4-e9eea836569a service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:16.195110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1219/2449] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:17.291921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22106c90-f883-4960-b871-947a22ebfcce None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:17.303989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22106c90-f883-4960-b871-947a22ebfcce None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:17.308852 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bd91f-03e5-4c6e-9491-7647f7de594f tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ffb2b9ef360438481973f6a9155b8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75da664260b1445b82f2ccd046c9023e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:17.309576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd91f-03e5-4c6e-9491-7647f7de594f tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:17.314746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd91f-03e5-4c6e-9491-7647f7de594f tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:17.314746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd91f-03e5-4c6e-9491-7647f7de594f tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:17.314746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bd91f-03e5-4c6e-9491-7647f7de594f tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:16:17.314746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bd91f-03e5-4c6e-9491-7647f7de594f tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:17.314746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bd91f-03e5-4c6e-9491-7647f7de594f tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:17.326601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1231/2450] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:16:17.331858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2304b-aed3-4912-98d4-e9eea836569a service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:17.342507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2304b-aed3-4912-98d4-e9eea836569a service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:17.348691 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231199b-ded6-4a7b-99b8-e739b65496a9 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ffb2b9ef360438481973f6a9155b8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75da664260b1445b82f2ccd046c9023e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:17.349434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231199b-ded6-4a7b-99b8-e739b65496a9 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:17.349731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231199b-ded6-4a7b-99b8-e739b65496a9 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:17.350017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231199b-ded6-4a7b-99b8-e739b65496a9 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:17.350759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231199b-ded6-4a7b-99b8-e739b65496a9 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:17.352688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231199b-ded6-4a7b-99b8-e739b65496a9 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:17.498457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1220/2451] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:17 2022] POST /identity/v3/projects => generated 352 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:17.503517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bd91f-03e5-4c6e-9491-7647f7de594f tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:17.516163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bd91f-03e5-4c6e-9491-7647f7de594f tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:17.529197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:17.602690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ea301-d37d-4787-9cb9-8ebfa1b4ab45 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ffb2b9ef360438481973f6a9155b8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75da664260b1445b82f2ccd046c9023e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:17.608230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ea301-d37d-4787-9cb9-8ebfa1b4ab45 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:17.608230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ea301-d37d-4787-9cb9-8ebfa1b4ab45 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:17.608230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ea301-d37d-4787-9cb9-8ebfa1b4ab45 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:17.608230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ea301-d37d-4787-9cb9-8ebfa1b4ab45 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:16:17.608230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ea301-d37d-4787-9cb9-8ebfa1b4ab45 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:17.609634 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ea301-d37d-4787-9cb9-8ebfa1b4ab45 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:17.619832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1232/2452] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:16:17.658057 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231199b-ded6-4a7b-99b8-e739b65496a9 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:17.669348 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231199b-ded6-4a7b-99b8-e739b65496a9 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:17.674704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b2102-6739-4a7b-a066-d904a8b9ed89 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ffb2b9ef360438481973f6a9155b8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75da664260b1445b82f2ccd046c9023e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:17.675712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b2102-6739-4a7b-a066-d904a8b9ed89 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:17.676090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b2102-6739-4a7b-a066-d904a8b9ed89 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:17.676465 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b2102-6739-4a7b-a066-d904a8b9ed89 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] PATH_INFO: `/v3/projects/61a2088446ae4db19a3941b8b569071e` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:17.677733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b2102-6739-4a7b-a066-d904a8b9ed89 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: Authorizing `identity:delete_project(project_id=61a2088446ae4db19a3941b8b569071e)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:17.679770 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b2102-6739-4a7b-a066-d904a8b9ed89 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:17.855809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=90,keystone:INSERT=6,keystone:DELETE=4 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:17.857351 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ea1b2102-6739-4a7b-a066-d904a8b9ed89 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a2088446ae4db19a3941b8b569071e', 'request_id': 'req-ea1b2102-6739-4a7b-a066-d904a8b9ed89'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:17.944202 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ea1b2102-6739-4a7b-a066-d904a8b9ed89 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a2088446ae4db19a3941b8b569071e', 'request_id': 'req-ea1b2102-6739-4a7b-a066-d904a8b9ed89'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:17.951339 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ea1b2102-6739-4a7b-a066-d904a8b9ed89 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a2088446ae4db19a3941b8b569071e', 'request_id': 'req-ea1b2102-6739-4a7b-a066-d904a8b9ed89'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:17.964762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ea1b2102-6739-4a7b-a066-d904a8b9ed89 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] The token cache is being invalidate because project 61a2088446ae4db19a3941b8b569071e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:17.965048 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ea1b2102-6739-4a7b-a066-d904a8b9ed89 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:17.965727 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ea1b2102-6739-4a7b-a066-d904a8b9ed89 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:17.967391 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1221/2453] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:17 2022] DELETE /identity/v3/projects/61a2088446ae4db19a3941b8b569071e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:18.797577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ea301-d37d-4787-9cb9-8ebfa1b4ab45 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:18.801126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-797ea301-d37d-4787-9cb9-8ebfa1b4ab45 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:18.900212 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ea301-d37d-4787-9cb9-8ebfa1b4ab45 tempest-VolumeQuotaClassesTest-700251001 tempest-VolumeQuotaClassesTest-700251001-project-admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:18.977447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:18.978720 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:18.979712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:18.979712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] PATH_INFO: `/v3/users/c0ced36a80e744d18022fa1c34fdf583` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:18.981540 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0ced36a80e744d18022fa1c34fdf583)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:18.983789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:19.230945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0ced36a80e744d18022fa1c34fdf583'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:19.231621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0ced36a80e744d18022fa1c34fdf583'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:19.238183 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:19.240381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0ced36a80e744d18022fa1c34fdf583'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:19.257431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0ced36a80e744d18022fa1c34fdf583'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:19.318173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0ced36a80e744d18022fa1c34fdf583'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:19.325103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1233/2454] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:18 2022] DELETE /identity/v3/users/c0ced36a80e744d18022fa1c34fdf583 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:19.330989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcff33-df73-49dd-b4a1-1c7af7cdf072 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:19.332119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcff33-df73-49dd-b4a1-1c7af7cdf072 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:19.332119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcff33-df73-49dd-b4a1-1c7af7cdf072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:19.352931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-2cbcff33-df73-49dd-b4a1-1c7af7cdf072 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:19.354156 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2cbcff33-df73-49dd-b4a1-1c7af7cdf072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:19.437993 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1222/2455] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:19.448299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:19.450554 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:19.520943 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0bdee-c235-4955-bb36-8f995e6d1a2a admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:19.525536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6183d-c44a-4636-8657-86b24c889c28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:19.526221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6183d-c44a-4636-8657-86b24c889c28 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:19.526569 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6183d-c44a-4636-8657-86b24c889c28 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:19.527232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6183d-c44a-4636-8657-86b24c889c28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:19.537751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6183d-c44a-4636-8657-86b24c889c28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:19.539284 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6183d-c44a-4636-8657-86b24c889c28 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:19.550141 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1234/2456] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:20.040722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcff33-df73-49dd-b4a1-1c7af7cdf072 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:20.042775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2cbcff33-df73-49dd-b4a1-1c7af7cdf072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:20.069747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcff33-df73-49dd-b4a1-1c7af7cdf072 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:20.081693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:20.081693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:20.081693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:20.081693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] PATH_INFO: `/v3/projects/9fb43314dce84f1c97fc18f89149c9b4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:20.082111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb43314dce84f1c97fc18f89149c9b4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:20.082796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:20.216862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb43314dce84f1c97fc18f89149c9b4', 'request_id': 'req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:20.292856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb43314dce84f1c97fc18f89149c9b4', 'request_id': 'req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:20.300248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb43314dce84f1c97fc18f89149c9b4', 'request_id': 'req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:20.312985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] The token cache is being invalidate because project 9fb43314dce84f1c97fc18f89149c9b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:20.312985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:20.314240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:20.315901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1223/2457] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:20 2022] DELETE /identity/v3/projects/9fb43314dce84f1c97fc18f89149c9b4 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:16:20.320257 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6183d-c44a-4636-8657-86b24c889c28 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:20.332246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6183d-c44a-4636-8657-86b24c889c28 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:20.394044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58dd4f-e02d-4d6b-a6cf-fa485d2ec976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:20.394973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58dd4f-e02d-4d6b-a6cf-fa485d2ec976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:20.395282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58dd4f-e02d-4d6b-a6cf-fa485d2ec976 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:20.395585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58dd4f-e02d-4d6b-a6cf-fa485d2ec976 admin admin] PATH_INFO: `/v3/users/2ffb2b9ef360438481973f6a9155b8a8` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:20.396823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58dd4f-e02d-4d6b-a6cf-fa485d2ec976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ffb2b9ef360438481973f6a9155b8a8)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:20.398907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58dd4f-e02d-4d6b-a6cf-fa485d2ec976 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:20.611378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b58dd4f-e02d-4d6b-a6cf-fa485d2ec976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ffb2b9ef360438481973f6a9155b8a8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:20.612310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b58dd4f-e02d-4d6b-a6cf-fa485d2ec976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ffb2b9ef360438481973f6a9155b8a8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:20.621274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b58dd4f-e02d-4d6b-a6cf-fa485d2ec976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ffb2b9ef360438481973f6a9155b8a8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:20.633730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b58dd4f-e02d-4d6b-a6cf-fa485d2ec976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ffb2b9ef360438481973f6a9155b8a8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:20.636365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:20.735978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b58dd4f-e02d-4d6b-a6cf-fa485d2ec976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ffb2b9ef360438481973f6a9155b8a8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:20.742938 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1235/2458] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:20 2022] DELETE /identity/v3/users/2ffb2b9ef360438481973f6a9155b8a8 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:21.176824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:21.178274 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:21.194285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee37c8-94c4-4eef-a7f2-e5aa50900fe2 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:21.269534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365c649-6e7c-4d67-b902-bfb0493f7f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:21.270497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365c649-6e7c-4d67-b902-bfb0493f7f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:21.270794 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365c649-6e7c-4d67-b902-bfb0493f7f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:21.271063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365c649-6e7c-4d67-b902-bfb0493f7f31 admin admin] PATH_INFO: `/v3/projects/75da664260b1445b82f2ccd046c9023e` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:21.272267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365c649-6e7c-4d67-b902-bfb0493f7f31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75da664260b1445b82f2ccd046c9023e)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:21.274428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365c649-6e7c-4d67-b902-bfb0493f7f31 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:21.276385 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:21.424123 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a365c649-6e7c-4d67-b902-bfb0493f7f31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75da664260b1445b82f2ccd046c9023e', 'request_id': 'req-a365c649-6e7c-4d67-b902-bfb0493f7f31'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:21.501927 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a365c649-6e7c-4d67-b902-bfb0493f7f31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75da664260b1445b82f2ccd046c9023e', 'request_id': 'req-a365c649-6e7c-4d67-b902-bfb0493f7f31'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:21.508118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a365c649-6e7c-4d67-b902-bfb0493f7f31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75da664260b1445b82f2ccd046c9023e', 'request_id': 'req-a365c649-6e7c-4d67-b902-bfb0493f7f31'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:21.520124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a365c649-6e7c-4d67-b902-bfb0493f7f31 admin admin] The token cache is being invalidate because project 75da664260b1445b82f2ccd046c9023e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:21.520414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a365c649-6e7c-4d67-b902-bfb0493f7f31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:21.521054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a365c649-6e7c-4d67-b902-bfb0493f7f31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:21.522657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1224/2459] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:21 2022] DELETE /identity/v3/projects/75da664260b1445b82f2ccd046c9023e => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:21.527972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75b303-541b-457f-bee7-6935f1701503 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:21.528499 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75b303-541b-457f-bee7-6935f1701503 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:21.528610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75b303-541b-457f-bee7-6935f1701503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:21.549114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-dd75b303-541b-457f-bee7-6935f1701503 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:21.550366 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-dd75b303-541b-457f-bee7-6935f1701503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:21.610589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1236/2460] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:16:21.618520 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422729e-dfe9-4ca9-a343-e4bbb8dff727 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:21.618814 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422729e-dfe9-4ca9-a343-e4bbb8dff727 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:21.619095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422729e-dfe9-4ca9-a343-e4bbb8dff727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:21.650616 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-1422729e-dfe9-4ca9-a343-e4bbb8dff727 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:21.651859 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1422729e-dfe9-4ca9-a343-e4bbb8dff727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:21.661092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1225/2461] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:21.665537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75b303-541b-457f-bee7-6935f1701503 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:21.675593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75b303-541b-457f-bee7-6935f1701503 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:21.680311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d94bb-80dc-414d-acf7-2d529d6282ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:21.680972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d94bb-80dc-414d-acf7-2d529d6282ad admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:21.681261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d94bb-80dc-414d-acf7-2d529d6282ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:21.681542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d94bb-80dc-414d-acf7-2d529d6282ad admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:21.682270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d94bb-80dc-414d-acf7-2d529d6282ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:16:21.682611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d94bb-80dc-414d-acf7-2d529d6282ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:21.684218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d94bb-80dc-414d-acf7-2d529d6282ad admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:21.707475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1237/2462] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:16:21.711876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422729e-dfe9-4ca9-a343-e4bbb8dff727 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:21.722889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422729e-dfe9-4ca9-a343-e4bbb8dff727 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:21.727931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb5617-714e-485e-b2be-ed7121dd33a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:21.728654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb5617-714e-485e-b2be-ed7121dd33a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:21.728946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb5617-714e-485e-b2be-ed7121dd33a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:21.729234 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb5617-714e-485e-b2be-ed7121dd33a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:21.730526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb5617-714e-485e-b2be-ed7121dd33a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:21.731957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb5617-714e-485e-b2be-ed7121dd33a3 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:21.851863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1226/2463] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:21 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:21.855840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d94bb-80dc-414d-acf7-2d529d6282ad admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:21.866082 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d94bb-80dc-414d-acf7-2d529d6282ad admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:21.925835 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756cad9-2b4b-4aeb-a2ed-38cd3cc227f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:21.926468 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756cad9-2b4b-4aeb-a2ed-38cd3cc227f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:21.926754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756cad9-2b4b-4aeb-a2ed-38cd3cc227f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:21.927031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756cad9-2b4b-4aeb-a2ed-38cd3cc227f0 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:21.927736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756cad9-2b4b-4aeb-a2ed-38cd3cc227f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:21.929679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756cad9-2b4b-4aeb-a2ed-38cd3cc227f0 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:22.061345 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1238/2464] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:21 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:22.066134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb5617-714e-485e-b2be-ed7121dd33a3 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:22.076590 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb5617-714e-485e-b2be-ed7121dd33a3 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:22.083011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0806f1-1b2c-488d-a6ba-99f1fa07d263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:22.083718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0806f1-1b2c-488d-a6ba-99f1fa07d263 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:22.083986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0806f1-1b2c-488d-a6ba-99f1fa07d263 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:22.084262 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0806f1-1b2c-488d-a6ba-99f1fa07d263 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:22.084990 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0806f1-1b2c-488d-a6ba-99f1fa07d263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:22.086679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0806f1-1b2c-488d-a6ba-99f1fa07d263 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:22.097774 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1227/2465] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:22.102177 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756cad9-2b4b-4aeb-a2ed-38cd3cc227f0 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:22.113965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756cad9-2b4b-4aeb-a2ed-38cd3cc227f0 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:22.118893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a19f1-9126-4ac7-9ee8-9dbf78849176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:22.119598 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a19f1-9126-4ac7-9ee8-9dbf78849176 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:22.119889 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a19f1-9126-4ac7-9ee8-9dbf78849176 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:22.120438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a19f1-9126-4ac7-9ee8-9dbf78849176 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:22.121178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a19f1-9126-4ac7-9ee8-9dbf78849176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:22.122852 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a19f1-9126-4ac7-9ee8-9dbf78849176 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:22.135335 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1239/2466] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:22.139558 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0806f1-1b2c-488d-a6ba-99f1fa07d263 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:22.151044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0806f1-1b2c-488d-a6ba-99f1fa07d263 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:22.155869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0650cdb-9977-4b45-aed0-d11713bc1adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:22.156876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0650cdb-9977-4b45-aed0-d11713bc1adc admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:22.157168 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0650cdb-9977-4b45-aed0-d11713bc1adc admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:22.157450 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0650cdb-9977-4b45-aed0-d11713bc1adc admin admin] PATH_INFO: `/v3/projects/d14616444c6c4af3907d82d4b6aba4b2/users/ca1343c69e9e4171a0a3daf1639b542e/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:22.173704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0650cdb-9977-4b45-aed0-d11713bc1adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14616444c6c4af3907d82d4b6aba4b2, user_id=ca1343c69e9e4171a0a3daf1639b542e, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:22.176452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0650cdb-9977-4b45-aed0-d11713bc1adc admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:22.267441 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1228/2467] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:22 2022] PUT /identity/v3/projects/d14616444c6c4af3907d82d4b6aba4b2/users/ca1343c69e9e4171a0a3daf1639b542e/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:22.279157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634705e-a47d-452c-9930-c15e72007475 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:22.279469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634705e-a47d-452c-9930-c15e72007475 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:22.279760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634705e-a47d-452c-9930-c15e72007475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:22.309800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-4634705e-a47d-452c-9930-c15e72007475 None None] MFA Rules not processed for user `ca1343c69e9e4171a0a3daf1639b542e`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:22.311248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-4634705e-a47d-452c-9930-c15e72007475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:22.364236 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:22.384250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1240/2468] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:22.388536 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0650cdb-9977-4b45-aed0-d11713bc1adc admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:22.461268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0650cdb-9977-4b45-aed0-d11713bc1adc admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:22.465909 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c5a17-6a6c-4083-89d5-0090f2befdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:22.466617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c5a17-6a6c-4083-89d5-0090f2befdf0 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:22.466892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c5a17-6a6c-4083-89d5-0090f2befdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:22.467164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c5a17-6a6c-4083-89d5-0090f2befdf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:22.467863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c5a17-6a6c-4083-89d5-0090f2befdf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:22.470597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c5a17-6a6c-4083-89d5-0090f2befdf0 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:22.564673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1229/2469] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:22 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:22.565951 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634705e-a47d-452c-9930-c15e72007475 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:22.577933 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634705e-a47d-452c-9930-c15e72007475 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:22.640652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccd1e5-d388-4a93-a4d5-1899d109fdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:22.641445 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccd1e5-d388-4a93-a4d5-1899d109fdb5 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:22.641734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccd1e5-d388-4a93-a4d5-1899d109fdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:22.642123 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccd1e5-d388-4a93-a4d5-1899d109fdb5 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:22.642870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccd1e5-d388-4a93-a4d5-1899d109fdb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:22.644807 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccd1e5-d388-4a93-a4d5-1899d109fdb5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:22.718898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1241/2470] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:22 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:22.720363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c5a17-6a6c-4083-89d5-0090f2befdf0 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:22.731979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c5a17-6a6c-4083-89d5-0090f2befdf0 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:22.737104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47759ee-6291-4c6f-ae79-9ce8e639c0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:22.737802 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47759ee-6291-4c6f-ae79-9ce8e639c0dd admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:22.738104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47759ee-6291-4c6f-ae79-9ce8e639c0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:22.738422 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47759ee-6291-4c6f-ae79-9ce8e639c0dd admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:22.739194 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47759ee-6291-4c6f-ae79-9ce8e639c0dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:22.740815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47759ee-6291-4c6f-ae79-9ce8e639c0dd admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:22.746861 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:22.753562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1230/2471] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:22.757674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccd1e5-d388-4a93-a4d5-1899d109fdb5 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:22.768653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccd1e5-d388-4a93-a4d5-1899d109fdb5 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:22.773666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c280e91-dbab-4547-b91f-b0738a5a14ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:22.774357 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c280e91-dbab-4547-b91f-b0738a5a14ca admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:22.774652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c280e91-dbab-4547-b91f-b0738a5a14ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:22.774929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c280e91-dbab-4547-b91f-b0738a5a14ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:22.775655 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c280e91-dbab-4547-b91f-b0738a5a14ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:22.777295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c280e91-dbab-4547-b91f-b0738a5a14ca admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:22.788406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1242/2472] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:22.793113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47759ee-6291-4c6f-ae79-9ce8e639c0dd admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:22.804673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47759ee-6291-4c6f-ae79-9ce8e639c0dd admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:22.809463 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fabbc-0ccb-4ae9-949f-78c10abc04cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:22.810440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fabbc-0ccb-4ae9-949f-78c10abc04cc admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:22.810762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fabbc-0ccb-4ae9-949f-78c10abc04cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:22.811048 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fabbc-0ccb-4ae9-949f-78c10abc04cc admin admin] PATH_INFO: `/v3/projects/3f99ae35abcb44e486cb9556ba171949/users/c73d72a7668d45d8be46ef10981d78ac/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:22.830659 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fabbc-0ccb-4ae9-949f-78c10abc04cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f99ae35abcb44e486cb9556ba171949, user_id=c73d72a7668d45d8be46ef10981d78ac, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:22.833361 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fabbc-0ccb-4ae9-949f-78c10abc04cc admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:22.916230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1231/2473] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:22 2022] PUT /identity/v3/projects/3f99ae35abcb44e486cb9556ba171949/users/c73d72a7668d45d8be46ef10981d78ac/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:22.927388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7bb7d-0741-4914-add1-8c7930bf3be9 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:22.927609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7bb7d-0741-4914-add1-8c7930bf3be9 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:22.927795 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7bb7d-0741-4914-add1-8c7930bf3be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:22.956717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-1de7bb7d-0741-4914-add1-8c7930bf3be9 None None] MFA Rules not processed for user `c73d72a7668d45d8be46ef10981d78ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:22.957866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1de7bb7d-0741-4914-add1-8c7930bf3be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:23.026809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1243/2474] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:23.031434 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fabbc-0ccb-4ae9-949f-78c10abc04cc admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:23.095533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fabbc-0ccb-4ae9-949f-78c10abc04cc admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:23.100389 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5c336-56cb-45ce-8006-64849135c77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:23.101067 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5c336-56cb-45ce-8006-64849135c77c admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:23.101381 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5c336-56cb-45ce-8006-64849135c77c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:23.101654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5c336-56cb-45ce-8006-64849135c77c admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:23.102462 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5c336-56cb-45ce-8006-64849135c77c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:23.104383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5c336-56cb-45ce-8006-64849135c77c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:23.203098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1232/2475] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:23 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:23.204692 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7bb7d-0741-4914-add1-8c7930bf3be9 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:23.216439 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7bb7d-0741-4914-add1-8c7930bf3be9 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:23.274727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:23.281669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ef395-ed0b-4972-b951-35a760e28a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:23.282461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ef395-ed0b-4972-b951-35a760e28a40 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:23.282758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ef395-ed0b-4972-b951-35a760e28a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:23.283039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ef395-ed0b-4972-b951-35a760e28a40 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:23.283839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ef395-ed0b-4972-b951-35a760e28a40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:23.286056 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ef395-ed0b-4972-b951-35a760e28a40 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:23.401797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1244/2476] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:23 2022] POST /identity/v3/users => generated 300 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:23.407605 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5c336-56cb-45ce-8006-64849135c77c admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:23.420621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5c336-56cb-45ce-8006-64849135c77c admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:23.427268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8cecc-f37c-4652-94a0-a9bd7a1ceecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:23.428092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8cecc-f37c-4652-94a0-a9bd7a1ceecb admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:23.428384 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8cecc-f37c-4652-94a0-a9bd7a1ceecb admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:23.428671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8cecc-f37c-4652-94a0-a9bd7a1ceecb admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:23.429421 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8cecc-f37c-4652-94a0-a9bd7a1ceecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:23.431087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8cecc-f37c-4652-94a0-a9bd7a1ceecb admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:23.443111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1233/2477] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:23.447400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ef395-ed0b-4972-b951-35a760e28a40 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:23.457878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ef395-ed0b-4972-b951-35a760e28a40 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:23.462574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff54c0-d522-43a3-a141-ebbdc470374e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:23.463464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff54c0-d522-43a3-a141-ebbdc470374e admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:23.463752 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff54c0-d522-43a3-a141-ebbdc470374e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:23.464030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff54c0-d522-43a3-a141-ebbdc470374e admin admin] PATH_INFO: `/v3/projects/2d58145dd6ce41cb94e5226ed19d1118/users/100a79c0844545f9b376f58f55562982/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:23.480788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff54c0-d522-43a3-a141-ebbdc470374e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d58145dd6ce41cb94e5226ed19d1118, user_id=100a79c0844545f9b376f58f55562982, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:23.483508 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff54c0-d522-43a3-a141-ebbdc470374e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:23.602529 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1245/2478] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:23 2022] PUT /identity/v3/projects/2d58145dd6ce41cb94e5226ed19d1118/users/100a79c0844545f9b376f58f55562982/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:16:23.606683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8cecc-f37c-4652-94a0-a9bd7a1ceecb admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:23.618267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8cecc-f37c-4652-94a0-a9bd7a1ceecb admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:23.679970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955a0c8-5c76-477a-925a-f861addb18d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:23.680653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955a0c8-5c76-477a-925a-f861addb18d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:23.680951 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955a0c8-5c76-477a-925a-f861addb18d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:23.681233 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955a0c8-5c76-477a-925a-f861addb18d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:23.681960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955a0c8-5c76-477a-925a-f861addb18d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:23.683617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955a0c8-5c76-477a-925a-f861addb18d2 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:23.694386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1234/2479] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:23.698733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff54c0-d522-43a3-a141-ebbdc470374e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:23.709384 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff54c0-d522-43a3-a141-ebbdc470374e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:23.714069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18eb105-4958-4d31-8cac-06503d52d975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:23.715004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18eb105-4958-4d31-8cac-06503d52d975 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:23.715299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18eb105-4958-4d31-8cac-06503d52d975 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:23.715611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18eb105-4958-4d31-8cac-06503d52d975 admin admin] PATH_INFO: `/v3/projects/2d58145dd6ce41cb94e5226ed19d1118/users/100a79c0844545f9b376f58f55562982/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:23.717615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18eb105-4958-4d31-8cac-06503d52d975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d58145dd6ce41cb94e5226ed19d1118, user_id=100a79c0844545f9b376f58f55562982, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:23.720558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18eb105-4958-4d31-8cac-06503d52d975 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:23.731351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1246/2480] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:23 2022] PUT /identity/v3/projects/2d58145dd6ce41cb94e5226ed19d1118/users/100a79c0844545f9b376f58f55562982/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:23.742460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737de4d3-f197-443f-9684-a3859b1bd062 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:23.747647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737de4d3-f197-443f-9684-a3859b1bd062 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:23.747647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737de4d3-f197-443f-9684-a3859b1bd062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:23.773220 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-737de4d3-f197-443f-9684-a3859b1bd062 None None] MFA Rules not processed for user `100a79c0844545f9b376f58f55562982`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:23.774492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-737de4d3-f197-443f-9684-a3859b1bd062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:23.809592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:23.846218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1235/2481] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:23.859505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18eb105-4958-4d31-8cac-06503d52d975 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:23.861450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d18eb105-4958-4d31-8cac-06503d52d975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:23.922728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18eb105-4958-4d31-8cac-06503d52d975 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:23.926505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f959b4-8832-40ce-93d5-59847d60a6d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:23.927171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f959b4-8832-40ce-93d5-59847d60a6d4 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:23.927462 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f959b4-8832-40ce-93d5-59847d60a6d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:23.927770 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f959b4-8832-40ce-93d5-59847d60a6d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:23.937960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f959b4-8832-40ce-93d5-59847d60a6d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:23.939456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f959b4-8832-40ce-93d5-59847d60a6d4 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:23.949227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1247/2482] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:24.843672 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737de4d3-f197-443f-9684-a3859b1bd062 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:24.846454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-737de4d3-f197-443f-9684-a3859b1bd062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:24.933013 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737de4d3-f197-443f-9684-a3859b1bd062 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:24.939415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feada26-41e1-4f5f-85cc-3faf6adf0cc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:24.940151 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feada26-41e1-4f5f-85cc-3faf6adf0cc6 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:24.940461 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feada26-41e1-4f5f-85cc-3faf6adf0cc6 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:24.940729 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feada26-41e1-4f5f-85cc-3faf6adf0cc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:24.942699 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8feada26-41e1-4f5f-85cc-3faf6adf0cc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:24.957782 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feada26-41e1-4f5f-85cc-3faf6adf0cc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:24.960575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feada26-41e1-4f5f-85cc-3faf6adf0cc6 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:24.973019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1236/2483] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:25.210101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f959b4-8832-40ce-93d5-59847d60a6d4 service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:25.211593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-08f959b4-8832-40ce-93d5-59847d60a6d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:25.223497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f959b4-8832-40ce-93d5-59847d60a6d4 service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:25.227486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d125fb-6516-4cfc-bae9-c062b7c8a271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:25.227808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d125fb-6516-4cfc-bae9-c062b7c8a271 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:25.228092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d125fb-6516-4cfc-bae9-c062b7c8a271 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:25.228395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d125fb-6516-4cfc-bae9-c062b7c8a271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:25.294558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d125fb-6516-4cfc-bae9-c062b7c8a271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:25.296208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d125fb-6516-4cfc-bae9-c062b7c8a271 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:25.306633 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1248/2484] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:32.521935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feada26-41e1-4f5f-85cc-3faf6adf0cc6 service nova] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:32.532442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feada26-41e1-4f5f-85cc-3faf6adf0cc6 service nova] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:32.536335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551f929-4e47-4a1e-adef-8d1df5087bee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:32.536978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551f929-4e47-4a1e-adef-8d1df5087bee service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:32.537267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551f929-4e47-4a1e-adef-8d1df5087bee service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:32.537546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551f929-4e47-4a1e-adef-8d1df5087bee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:32.596552 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551f929-4e47-4a1e-adef-8d1df5087bee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:32.598215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551f929-4e47-4a1e-adef-8d1df5087bee service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:32.609092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1237/2485] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:35.301806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=72,keystone:INSERT=5 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:36.508680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d125fb-6516-4cfc-bae9-c062b7c8a271 service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:36.574153 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d125fb-6516-4cfc-bae9-c062b7c8a271 service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:36.579390 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:36.583794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:36.583794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:36.583794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] PATH_INFO: `/v3/users/ca1343c69e9e4171a0a3daf1639b542e` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:36.583794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca1343c69e9e4171a0a3daf1639b542e)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:36.584266 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:36.778115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca1343c69e9e4171a0a3daf1639b542e'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:36.778924 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca1343c69e9e4171a0a3daf1639b542e'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:36.788952 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca1343c69e9e4171a0a3daf1639b542e'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:36.801782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca1343c69e9e4171a0a3daf1639b542e'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:36.866144 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca1343c69e9e4171a0a3daf1639b542e'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:36.872983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1249/2486] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:36 2022] DELETE /identity/v3/users/ca1343c69e9e4171a0a3daf1639b542e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:36.877822 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bc9d8-91f4-429a-a1a9-2e2f5c8b2906 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:36.878380 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bc9d8-91f4-429a-a1a9-2e2f5c8b2906 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:36.878506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bc9d8-91f4-429a-a1a9-2e2f5c8b2906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:36.900572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-702bc9d8-91f4-429a-a1a9-2e2f5c8b2906 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:36.901793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-702bc9d8-91f4-429a-a1a9-2e2f5c8b2906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:36.949383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:36.960699 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1238/2487] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:36.971227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:36.973131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:37.031125 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dcc0f3-8942-44a4-b6e6-ccfb50b0cd31 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:37.034980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74f3d9-3599-48e7-8982-3896a1babfcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:37.035665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74f3d9-3599-48e7-8982-3896a1babfcb service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:37.035954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74f3d9-3599-48e7-8982-3896a1babfcb service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:37.036234 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74f3d9-3599-48e7-8982-3896a1babfcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:37.045675 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74f3d9-3599-48e7-8982-3896a1babfcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:37.048068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74f3d9-3599-48e7-8982-3896a1babfcb service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:37.057919 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1250/2488] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:37.506275 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702bc9d8-91f4-429a-a1a9-2e2f5c8b2906 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:37.507764 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-702bc9d8-91f4-429a-a1a9-2e2f5c8b2906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:37.522995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702bc9d8-91f4-429a-a1a9-2e2f5c8b2906 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:37.528171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:37.529054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:37.529342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:37.529624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] PATH_INFO: `/v3/projects/d14616444c6c4af3907d82d4b6aba4b2` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:37.531064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14616444c6c4af3907d82d4b6aba4b2)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:37.533282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:37.649446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14616444c6c4af3907d82d4b6aba4b2', 'request_id': 'req-93fa771f-ca74-454e-8779-e1b1365f5abf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:37.716963 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14616444c6c4af3907d82d4b6aba4b2', 'request_id': 'req-93fa771f-ca74-454e-8779-e1b1365f5abf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:37.723283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14616444c6c4af3907d82d4b6aba4b2', 'request_id': 'req-93fa771f-ca74-454e-8779-e1b1365f5abf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:37.735105 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] The token cache is being invalidate because project d14616444c6c4af3907d82d4b6aba4b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:37.735400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:37.736065 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:37.737629 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1239/2489] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:37 2022] DELETE /identity/v3/projects/d14616444c6c4af3907d82d4b6aba4b2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:16:37.742263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74f3d9-3599-48e7-8982-3896a1babfcb service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:37.754106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74f3d9-3599-48e7-8982-3896a1babfcb service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:37.774118 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:37.817425 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:37.818221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:37.818535 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:37.818820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] PATH_INFO: `/v3/users/c73d72a7668d45d8be46ef10981d78ac` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:37.820035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c73d72a7668d45d8be46ef10981d78ac)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:37.821944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:38.013436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c73d72a7668d45d8be46ef10981d78ac'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:38.014104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c73d72a7668d45d8be46ef10981d78ac'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:38.021800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c73d72a7668d45d8be46ef10981d78ac'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:38.033919 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c73d72a7668d45d8be46ef10981d78ac'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:38.094796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c73d72a7668d45d8be46ef10981d78ac'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:38.101925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1251/2490] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:37 2022] DELETE /identity/v3/users/c73d72a7668d45d8be46ef10981d78ac => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:38.494626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:38.500986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:38.510230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa771f-ca74-454e-8779-e1b1365f5abf admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:38.570695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:38.571576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:38.571866 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:38.572149 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] PATH_INFO: `/v3/projects/3f99ae35abcb44e486cb9556ba171949` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:38.573347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f99ae35abcb44e486cb9556ba171949)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:38.575361 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:38.661746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f99ae35abcb44e486cb9556ba171949', 'request_id': 'req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:38.709845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f99ae35abcb44e486cb9556ba171949', 'request_id': 'req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:38.716071 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f99ae35abcb44e486cb9556ba171949', 'request_id': 'req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:38.729219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] The token cache is being invalidate because project 3f99ae35abcb44e486cb9556ba171949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:38.729526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:38.730181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:38.731830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1240/2491] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:38 2022] DELETE /identity/v3/projects/3f99ae35abcb44e486cb9556ba171949 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:38.736143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:38.737618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:38.751323 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab6f07-fc20-467e-aac5-624b999c0367 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:38.812502 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f122a-bf75-46b7-ae42-5bc68efd13f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:38.813382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f122a-bf75-46b7-ae42-5bc68efd13f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:38.813663 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f122a-bf75-46b7-ae42-5bc68efd13f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:38.813966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f122a-bf75-46b7-ae42-5bc68efd13f3 admin admin] PATH_INFO: `/v3/users/100a79c0844545f9b376f58f55562982` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:38.815197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f122a-bf75-46b7-ae42-5bc68efd13f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=100a79c0844545f9b376f58f55562982)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:38.817140 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f122a-bf75-46b7-ae42-5bc68efd13f3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:38.975386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:38.978223 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a96f122a-bf75-46b7-ae42-5bc68efd13f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100a79c0844545f9b376f58f55562982'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:38.979128 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a96f122a-bf75-46b7-ae42-5bc68efd13f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100a79c0844545f9b376f58f55562982'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:38.987059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a96f122a-bf75-46b7-ae42-5bc68efd13f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100a79c0844545f9b376f58f55562982'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:38.999618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a96f122a-bf75-46b7-ae42-5bc68efd13f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100a79c0844545f9b376f58f55562982'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:39.075275 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-a96f122a-bf75-46b7-ae42-5bc68efd13f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100a79c0844545f9b376f58f55562982'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:39.081982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1252/2492] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:38 2022] DELETE /identity/v3/users/100a79c0844545f9b376f58f55562982 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:39.553238 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:39.554772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:39.569998 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3edca-22a7-4b69-8e6f-0c0503f80f63 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:39.576407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:39.635754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52584dbf-21e0-43dc-ae04-e0cb7265f597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:39.636751 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52584dbf-21e0-43dc-ae04-e0cb7265f597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:39.637032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52584dbf-21e0-43dc-ae04-e0cb7265f597 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:39.637312 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52584dbf-21e0-43dc-ae04-e0cb7265f597 admin admin] PATH_INFO: `/v3/projects/2d58145dd6ce41cb94e5226ed19d1118` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:39.638567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52584dbf-21e0-43dc-ae04-e0cb7265f597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d58145dd6ce41cb94e5226ed19d1118)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:39.641232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52584dbf-21e0-43dc-ae04-e0cb7265f597 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:39.740562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-52584dbf-21e0-43dc-ae04-e0cb7265f597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d58145dd6ce41cb94e5226ed19d1118', 'request_id': 'req-52584dbf-21e0-43dc-ae04-e0cb7265f597'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:39.837626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-52584dbf-21e0-43dc-ae04-e0cb7265f597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d58145dd6ce41cb94e5226ed19d1118', 'request_id': 'req-52584dbf-21e0-43dc-ae04-e0cb7265f597'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:39.844639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-52584dbf-21e0-43dc-ae04-e0cb7265f597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d58145dd6ce41cb94e5226ed19d1118', 'request_id': 'req-52584dbf-21e0-43dc-ae04-e0cb7265f597'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:39.859856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-52584dbf-21e0-43dc-ae04-e0cb7265f597 admin admin] The token cache is being invalidate because project 2d58145dd6ce41cb94e5226ed19d1118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:39.860693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-52584dbf-21e0-43dc-ae04-e0cb7265f597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:39.860973 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-52584dbf-21e0-43dc-ae04-e0cb7265f597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:39.862636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1241/2493] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:39 2022] DELETE /identity/v3/projects/2d58145dd6ce41cb94e5226ed19d1118 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:39.869431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0a9d1-15a8-4deb-9c25-a48f01e03249 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:39.870270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0a9d1-15a8-4deb-9c25-a48f01e03249 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:39.870270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0a9d1-15a8-4deb-9c25-a48f01e03249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:39.894921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-e0c0a9d1-15a8-4deb-9c25-a48f01e03249 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:39.896371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e0c0a9d1-15a8-4deb-9c25-a48f01e03249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:39.960404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1253/2494] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:39.969163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924408a-e67a-483a-8f8f-d6915283c748 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:39.969538 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924408a-e67a-483a-8f8f-d6915283c748 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:39.972655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924408a-e67a-483a-8f8f-d6915283c748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:39.990449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-9924408a-e67a-483a-8f8f-d6915283c748 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:39.991591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9924408a-e67a-483a-8f8f-d6915283c748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:40.000526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1242/2495] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:40.004856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0a9d1-15a8-4deb-9c25-a48f01e03249 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:40.015408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0a9d1-15a8-4deb-9c25-a48f01e03249 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:40.020606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b02b07-6f3a-4674-ad3b-7adcb578ce79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:40.021267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b02b07-6f3a-4674-ad3b-7adcb578ce79 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:40.021553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b02b07-6f3a-4674-ad3b-7adcb578ce79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:40.021835 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b02b07-6f3a-4674-ad3b-7adcb578ce79 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:40.022595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b02b07-6f3a-4674-ad3b-7adcb578ce79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:16:40.022904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b02b07-6f3a-4674-ad3b-7adcb578ce79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:40.024498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b02b07-6f3a-4674-ad3b-7adcb578ce79 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:40.038943 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1254/2496] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:16:40.043549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924408a-e67a-483a-8f8f-d6915283c748 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:40.054695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924408a-e67a-483a-8f8f-d6915283c748 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:40.059886 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44773000-f69a-4a0f-95cb-b75eedcab411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:40.060619 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44773000-f69a-4a0f-95cb-b75eedcab411 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:40.060943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44773000-f69a-4a0f-95cb-b75eedcab411 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:40.061226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44773000-f69a-4a0f-95cb-b75eedcab411 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:40.062406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44773000-f69a-4a0f-95cb-b75eedcab411 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:40.064833 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44773000-f69a-4a0f-95cb-b75eedcab411 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:40.169810 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1243/2497] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:40 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:40.174311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b02b07-6f3a-4674-ad3b-7adcb578ce79 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:40.185604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b02b07-6f3a-4674-ad3b-7adcb578ce79 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:40.249590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04af522-acd6-470c-abc5-099bb4226395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:40.250013 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04af522-acd6-470c-abc5-099bb4226395 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:40.250613 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04af522-acd6-470c-abc5-099bb4226395 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:40.250863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04af522-acd6-470c-abc5-099bb4226395 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:40.251594 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04af522-acd6-470c-abc5-099bb4226395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:40.253539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04af522-acd6-470c-abc5-099bb4226395 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:40.335874 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1255/2498] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:40 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:40.339999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44773000-f69a-4a0f-95cb-b75eedcab411 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:40.351609 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44773000-f69a-4a0f-95cb-b75eedcab411 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:40.356276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff5092-4cd7-4768-aada-b3c35a1874ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:40.356946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff5092-4cd7-4768-aada-b3c35a1874ee admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:40.357231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff5092-4cd7-4768-aada-b3c35a1874ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:40.357508 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff5092-4cd7-4768-aada-b3c35a1874ee admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:40.358302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff5092-4cd7-4768-aada-b3c35a1874ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:40.360202 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff5092-4cd7-4768-aada-b3c35a1874ee admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:40.370798 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1244/2499] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:40.375422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04af522-acd6-470c-abc5-099bb4226395 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:40.386052 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04af522-acd6-470c-abc5-099bb4226395 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:40.391509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2b5d9-1a9f-41f3-8514-326ff371f9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:40.392204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2b5d9-1a9f-41f3-8514-326ff371f9e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:40.392672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2b5d9-1a9f-41f3-8514-326ff371f9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:40.392960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2b5d9-1a9f-41f3-8514-326ff371f9e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:40.393713 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2b5d9-1a9f-41f3-8514-326ff371f9e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:40.396094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2b5d9-1a9f-41f3-8514-326ff371f9e3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:40.409712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1256/2500] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:40.413813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff5092-4cd7-4768-aada-b3c35a1874ee admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:40.424447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff5092-4cd7-4768-aada-b3c35a1874ee admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:40.431453 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d764ba-ab6c-4187-8d8a-d5b58a4fa1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:40.432363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d764ba-ab6c-4187-8d8a-d5b58a4fa1b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:40.432646 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d764ba-ab6c-4187-8d8a-d5b58a4fa1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:40.432920 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d764ba-ab6c-4187-8d8a-d5b58a4fa1b4 admin admin] PATH_INFO: `/v3/projects/a0ef43661f054dceb41f26dfdde4d81a/users/05afe853daf6436aa83077bd7ea3ec98/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:40.448832 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d764ba-ab6c-4187-8d8a-d5b58a4fa1b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ef43661f054dceb41f26dfdde4d81a, user_id=05afe853daf6436aa83077bd7ea3ec98, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:40.451528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d764ba-ab6c-4187-8d8a-d5b58a4fa1b4 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:40.527663 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1245/2501] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:40 2022] PUT /identity/v3/projects/a0ef43661f054dceb41f26dfdde4d81a/users/05afe853daf6436aa83077bd7ea3ec98/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:40.539434 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa649f-cb2d-45c3-83b5-e3f1336fcc4c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:40.539730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa649f-cb2d-45c3-83b5-e3f1336fcc4c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:40.540011 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa649f-cb2d-45c3-83b5-e3f1336fcc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:40.568869 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-8baa649f-cb2d-45c3-83b5-e3f1336fcc4c None None] MFA Rules not processed for user `05afe853daf6436aa83077bd7ea3ec98`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:40.570094 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8baa649f-cb2d-45c3-83b5-e3f1336fcc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:40.584179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:40.637158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1257/2502] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:40.642587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d764ba-ab6c-4187-8d8a-d5b58a4fa1b4 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:40.682261 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:40.708473 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d764ba-ab6c-4187-8d8a-d5b58a4fa1b4 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:40.714116 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcd437-d7ed-4bda-9de7-ac50247a8c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:40.714854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcd437-d7ed-4bda-9de7-ac50247a8c96 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:40.715162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcd437-d7ed-4bda-9de7-ac50247a8c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:40.715447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcd437-d7ed-4bda-9de7-ac50247a8c96 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:40.716159 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcd437-d7ed-4bda-9de7-ac50247a8c96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:40.718354 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcd437-d7ed-4bda-9de7-ac50247a8c96 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:40.874772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1246/2503] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:40 2022] POST /identity/v3/projects => generated 381 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:40.878793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa649f-cb2d-45c3-83b5-e3f1336fcc4c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:40.890642 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa649f-cb2d-45c3-83b5-e3f1336fcc4c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:40.951620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a9e9c-9565-4728-a963-694903485f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:40.952251 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a9e9c-9565-4728-a963-694903485f3b admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:40.952595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a9e9c-9565-4728-a963-694903485f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:40.952956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a9e9c-9565-4728-a963-694903485f3b admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:40.953712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a9e9c-9565-4728-a963-694903485f3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:40.955679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a9e9c-9565-4728-a963-694903485f3b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:41.048302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1258/2504] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:40 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:41.052345 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcd437-d7ed-4bda-9de7-ac50247a8c96 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:41.063471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcd437-d7ed-4bda-9de7-ac50247a8c96 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:41.068857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4631ad7-9c27-4ba1-81d0-4224d1684801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:41.069556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4631ad7-9c27-4ba1-81d0-4224d1684801 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:41.069843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4631ad7-9c27-4ba1-81d0-4224d1684801 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:41.070124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4631ad7-9c27-4ba1-81d0-4224d1684801 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:41.070958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4631ad7-9c27-4ba1-81d0-4224d1684801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:41.072644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4631ad7-9c27-4ba1-81d0-4224d1684801 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:41.083591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1247/2505] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:41.087932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a9e9c-9565-4728-a963-694903485f3b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:41.099862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a9e9c-9565-4728-a963-694903485f3b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:41.104605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693a202-65c7-4eac-bded-35d3d8dec3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:41.105503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693a202-65c7-4eac-bded-35d3d8dec3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:41.105793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693a202-65c7-4eac-bded-35d3d8dec3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:41.106075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693a202-65c7-4eac-bded-35d3d8dec3f7 admin admin] PATH_INFO: `/v3/projects/a0f57a3c806e4be0ace9216655fbf5df/users/5158c7f6d6614c3d971b12e8bc8936cf/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:41.122124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9693a202-65c7-4eac-bded-35d3d8dec3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f57a3c806e4be0ace9216655fbf5df, user_id=5158c7f6d6614c3d971b12e8bc8936cf, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:41.125117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9693a202-65c7-4eac-bded-35d3d8dec3f7 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:41.225342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1259/2506] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:41 2022] PUT /identity/v3/projects/a0f57a3c806e4be0ace9216655fbf5df/users/5158c7f6d6614c3d971b12e8bc8936cf/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:41.229306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4631ad7-9c27-4ba1-81d0-4224d1684801 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:41.240167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4631ad7-9c27-4ba1-81d0-4224d1684801 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:41.304342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172e99c-6530-4a44-821f-525d9ea1d253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:41.305034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172e99c-6530-4a44-821f-525d9ea1d253 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:41.305324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172e99c-6530-4a44-821f-525d9ea1d253 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:41.305603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172e99c-6530-4a44-821f-525d9ea1d253 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:41.306350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172e99c-6530-4a44-821f-525d9ea1d253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:41.307966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172e99c-6530-4a44-821f-525d9ea1d253 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:41.324112 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1248/2507] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:41.325668 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693a202-65c7-4eac-bded-35d3d8dec3f7 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:41.338471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693a202-65c7-4eac-bded-35d3d8dec3f7 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:41.343429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26a8c2-484d-4196-a746-f21bfcb1eeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:41.344766 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26a8c2-484d-4196-a746-f21bfcb1eeda admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:41.345079 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26a8c2-484d-4196-a746-f21bfcb1eeda admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:41.345340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26a8c2-484d-4196-a746-f21bfcb1eeda admin admin] PATH_INFO: `/v3/projects/a0f57a3c806e4be0ace9216655fbf5df/users/5158c7f6d6614c3d971b12e8bc8936cf/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:41.348291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26a8c2-484d-4196-a746-f21bfcb1eeda admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f57a3c806e4be0ace9216655fbf5df, user_id=5158c7f6d6614c3d971b12e8bc8936cf, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:41.352695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26a8c2-484d-4196-a746-f21bfcb1eeda admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:41.365515 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1260/2508] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:41 2022] PUT /identity/v3/projects/a0f57a3c806e4be0ace9216655fbf5df/users/5158c7f6d6614c3d971b12e8bc8936cf/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:41.384502 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869e98a-b418-4f05-aabe-7e9d28004fe2 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:41.384764 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869e98a-b418-4f05-aabe-7e9d28004fe2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:41.385053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869e98a-b418-4f05-aabe-7e9d28004fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:41.418579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-5869e98a-b418-4f05-aabe-7e9d28004fe2 None None] MFA Rules not processed for user `5158c7f6d6614c3d971b12e8bc8936cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:41.419830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5869e98a-b418-4f05-aabe-7e9d28004fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:41.495325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1249/2509] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:41.507383 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26a8c2-484d-4196-a746-f21bfcb1eeda admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:41.509218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fe26a8c2-484d-4196-a746-f21bfcb1eeda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:41.569395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26a8c2-484d-4196-a746-f21bfcb1eeda admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:41.573168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e1128-25c1-4a87-be04-1fc5152ca4ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:41.573819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e1128-25c1-4a87-be04-1fc5152ca4ce service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:41.574103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e1128-25c1-4a87-be04-1fc5152ca4ce service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:41.574406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e1128-25c1-4a87-be04-1fc5152ca4ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:41.584960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e1128-25c1-4a87-be04-1fc5152ca4ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:41.586455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e1128-25c1-4a87-be04-1fc5152ca4ce service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:41.596697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1261/2510] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:42.783042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869e98a-b418-4f05-aabe-7e9d28004fe2 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:42.822645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:42.854308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869e98a-b418-4f05-aabe-7e9d28004fe2 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:42.859732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:42.860640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:42.860970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:42.861283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] PATH_INFO: `/v3/users/05afe853daf6436aa83077bd7ea3ec98` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:42.862530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05afe853daf6436aa83077bd7ea3ec98)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:42.867604 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:43.055464 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05afe853daf6436aa83077bd7ea3ec98'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:43.056128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05afe853daf6436aa83077bd7ea3ec98'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:43.056749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05afe853daf6436aa83077bd7ea3ec98'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:43.126792 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05afe853daf6436aa83077bd7ea3ec98'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:43.141290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05afe853daf6436aa83077bd7ea3ec98'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:43.153866 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1250/2511] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:42 2022] DELETE /identity/v3/users/05afe853daf6436aa83077bd7ea3ec98 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:43.158733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc0b05-53e8-4e5a-b691-adc5c0ff3f02 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:43.159222 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc0b05-53e8-4e5a-b691-adc5c0ff3f02 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:43.159347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc0b05-53e8-4e5a-b691-adc5c0ff3f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:43.179682 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-84cc0b05-53e8-4e5a-b691-adc5c0ff3f02 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:43.180902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-84cc0b05-53e8-4e5a-b691-adc5c0ff3f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:43.186030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:43.245755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1262/2512] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:43.256357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:43.257790 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:43.317568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c462e-1c09-4369-ad04-ed2473cc9469 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:43.321297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0039e-bb69-4c46-82c5-3acca14fa98d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:43.321867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0039e-bb69-4c46-82c5-3acca14fa98d service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:43.322157 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0039e-bb69-4c46-82c5-3acca14fa98d service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:43.322477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0039e-bb69-4c46-82c5-3acca14fa98d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:43.332324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0039e-bb69-4c46-82c5-3acca14fa98d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:43.333785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0039e-bb69-4c46-82c5-3acca14fa98d service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:43.343595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1251/2513] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:43.781274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc0b05-53e8-4e5a-b691-adc5c0ff3f02 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:43.783001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-84cc0b05-53e8-4e5a-b691-adc5c0ff3f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:43.798134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc0b05-53e8-4e5a-b691-adc5c0ff3f02 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:43.822827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:43.822827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:43.822827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:43.822827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] PATH_INFO: `/v3/projects/a0ef43661f054dceb41f26dfdde4d81a` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:43.824012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0ef43661f054dceb41f26dfdde4d81a)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:43.824012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:44.056086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0ef43661f054dceb41f26dfdde4d81a', 'request_id': 'req-027934c0-7357-4923-881e-77d3771645c2'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:44.063353 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0ef43661f054dceb41f26dfdde4d81a', 'request_id': 'req-027934c0-7357-4923-881e-77d3771645c2'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:44.069230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0ef43661f054dceb41f26dfdde4d81a', 'request_id': 'req-027934c0-7357-4923-881e-77d3771645c2'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:44.141113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] The token cache is being invalidate because project a0ef43661f054dceb41f26dfdde4d81a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:44.141113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:44.141611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:44.143273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1263/2514] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:43 2022] DELETE /identity/v3/projects/a0ef43661f054dceb41f26dfdde4d81a => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:44.147446 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0039e-bb69-4c46-82c5-3acca14fa98d service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:44.161492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0039e-bb69-4c46-82c5-3acca14fa98d service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:44.212745 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:44.221728 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90194405-e578-4701-9377-26d00695b268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:44.222649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90194405-e578-4701-9377-26d00695b268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:44.222940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90194405-e578-4701-9377-26d00695b268 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:44.223256 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90194405-e578-4701-9377-26d00695b268 admin admin] PATH_INFO: `/v3/users/5158c7f6d6614c3d971b12e8bc8936cf` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:44.224472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90194405-e578-4701-9377-26d00695b268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5158c7f6d6614c3d971b12e8bc8936cf)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:44.226472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90194405-e578-4701-9377-26d00695b268 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:44.445664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-90194405-e578-4701-9377-26d00695b268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5158c7f6d6614c3d971b12e8bc8936cf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:44.446668 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-90194405-e578-4701-9377-26d00695b268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5158c7f6d6614c3d971b12e8bc8936cf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:44.447452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-90194405-e578-4701-9377-26d00695b268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5158c7f6d6614c3d971b12e8bc8936cf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:44.522955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-90194405-e578-4701-9377-26d00695b268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5158c7f6d6614c3d971b12e8bc8936cf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:44.536416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-90194405-e578-4701-9377-26d00695b268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5158c7f6d6614c3d971b12e8bc8936cf'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:44.550279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1252/2515] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:44 2022] DELETE /identity/v3/users/5158c7f6d6614c3d971b12e8bc8936cf => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:44.977171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:44.978689 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:44.991902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027934c0-7357-4923-881e-77d3771645c2 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:45.063671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591b3a0-b307-490d-8069-7cee520bbf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:45.064566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591b3a0-b307-490d-8069-7cee520bbf39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:45.064851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591b3a0-b307-490d-8069-7cee520bbf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:45.065126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591b3a0-b307-490d-8069-7cee520bbf39 admin admin] PATH_INFO: `/v3/projects/a0f57a3c806e4be0ace9216655fbf5df` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:45.066340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591b3a0-b307-490d-8069-7cee520bbf39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f57a3c806e4be0ace9216655fbf5df)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:45.068272 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591b3a0-b307-490d-8069-7cee520bbf39 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:45.167525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:45.176541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2591b3a0-b307-490d-8069-7cee520bbf39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f57a3c806e4be0ace9216655fbf5df', 'request_id': 'req-2591b3a0-b307-490d-8069-7cee520bbf39'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:45.182741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2591b3a0-b307-490d-8069-7cee520bbf39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f57a3c806e4be0ace9216655fbf5df', 'request_id': 'req-2591b3a0-b307-490d-8069-7cee520bbf39'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:45.189219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2591b3a0-b307-490d-8069-7cee520bbf39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f57a3c806e4be0ace9216655fbf5df', 'request_id': 'req-2591b3a0-b307-490d-8069-7cee520bbf39'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:45.277169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2591b3a0-b307-490d-8069-7cee520bbf39 admin admin] The token cache is being invalidate because project a0f57a3c806e4be0ace9216655fbf5df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:45.277464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2591b3a0-b307-490d-8069-7cee520bbf39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:45.278102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-2591b3a0-b307-490d-8069-7cee520bbf39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:45.279691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1264/2516] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:44 2022] DELETE /identity/v3/projects/a0f57a3c806e4be0ace9216655fbf5df => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:45.284870 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56eb04c-49e8-4568-b55f-30544c55502d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:45.285355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56eb04c-49e8-4568-b55f-30544c55502d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:45.285475 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56eb04c-49e8-4568-b55f-30544c55502d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:45.305613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e56eb04c-49e8-4568-b55f-30544c55502d None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:45.306856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e56eb04c-49e8-4568-b55f-30544c55502d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:45.372330 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1253/2517] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:45.380854 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd11aaf-6ca7-4044-8fbd-c3a0303d2cb8 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:45.381146 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd11aaf-6ca7-4044-8fbd-c3a0303d2cb8 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:45.381423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd11aaf-6ca7-4044-8fbd-c3a0303d2cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:45.403199 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-dcd11aaf-6ca7-4044-8fbd-c3a0303d2cb8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:45.404428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-dcd11aaf-6ca7-4044-8fbd-c3a0303d2cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:45.412952 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1265/2518] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:45.417482 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56eb04c-49e8-4568-b55f-30544c55502d None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:45.428415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56eb04c-49e8-4568-b55f-30544c55502d None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:45.433965 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba29a81-cb7c-4afa-83bc-f42c63a86429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:45.434612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba29a81-cb7c-4afa-83bc-f42c63a86429 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:45.434906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba29a81-cb7c-4afa-83bc-f42c63a86429 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:45.435190 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba29a81-cb7c-4afa-83bc-f42c63a86429 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:45.435930 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba29a81-cb7c-4afa-83bc-f42c63a86429 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:16:45.436885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba29a81-cb7c-4afa-83bc-f42c63a86429 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:45.438651 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba29a81-cb7c-4afa-83bc-f42c63a86429 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:45.450936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1254/2519] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:16:45.455736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd11aaf-6ca7-4044-8fbd-c3a0303d2cb8 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:45.466852 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd11aaf-6ca7-4044-8fbd-c3a0303d2cb8 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:45.473605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190f851-4eb4-46ce-9ffb-b0202bca6c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:45.474299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190f851-4eb4-46ce-9ffb-b0202bca6c98 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:45.474620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190f851-4eb4-46ce-9ffb-b0202bca6c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:45.474898 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190f851-4eb4-46ce-9ffb-b0202bca6c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:45.475607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190f851-4eb4-46ce-9ffb-b0202bca6c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:45.477519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190f851-4eb4-46ce-9ffb-b0202bca6c98 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:45.565802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1266/2520] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:45 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:16:45.570152 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba29a81-cb7c-4afa-83bc-f42c63a86429 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:45.582062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba29a81-cb7c-4afa-83bc-f42c63a86429 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:45.642458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddf4ae-ab09-4832-ad17-f9feade04e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:45.643043 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddf4ae-ab09-4832-ad17-f9feade04e63 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:45.643338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddf4ae-ab09-4832-ad17-f9feade04e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:45.643624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddf4ae-ab09-4832-ad17-f9feade04e63 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:45.644326 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddf4ae-ab09-4832-ad17-f9feade04e63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:45.646261 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddf4ae-ab09-4832-ad17-f9feade04e63 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:45.654235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:45.763486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1255/2521] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:45 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:45.768407 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190f851-4eb4-46ce-9ffb-b0202bca6c98 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:45.779147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190f851-4eb4-46ce-9ffb-b0202bca6c98 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:45.784652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85251f-84b8-4318-a062-b3b67802c480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:45.785327 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85251f-84b8-4318-a062-b3b67802c480 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:45.785618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85251f-84b8-4318-a062-b3b67802c480 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:45.785900 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85251f-84b8-4318-a062-b3b67802c480 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:45.786667 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85251f-84b8-4318-a062-b3b67802c480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:45.788270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85251f-84b8-4318-a062-b3b67802c480 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:45.802455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1267/2522] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:45.811023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddf4ae-ab09-4832-ad17-f9feade04e63 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:45.821608 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddf4ae-ab09-4832-ad17-f9feade04e63 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:45.826982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016566c-6943-47fa-bf02-7ba62b89b84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:45.827703 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016566c-6943-47fa-bf02-7ba62b89b84a admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:45.827993 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016566c-6943-47fa-bf02-7ba62b89b84a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:45.828273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016566c-6943-47fa-bf02-7ba62b89b84a admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:45.829001 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016566c-6943-47fa-bf02-7ba62b89b84a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:45.830656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016566c-6943-47fa-bf02-7ba62b89b84a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:45.841596 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1256/2523] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:45.846765 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85251f-84b8-4318-a062-b3b67802c480 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:45.857480 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85251f-84b8-4318-a062-b3b67802c480 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:45.863295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4de19-f79c-48b2-959f-4b486755b9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:45.864195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4de19-f79c-48b2-959f-4b486755b9b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:45.864474 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4de19-f79c-48b2-959f-4b486755b9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:45.864752 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4de19-f79c-48b2-959f-4b486755b9b6 admin admin] PATH_INFO: `/v3/projects/85f00d3c21e34397902dcf2e69b18870/users/c10745983f34485d86ae13dd09b5fa02/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:45.884979 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4de19-f79c-48b2-959f-4b486755b9b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85f00d3c21e34397902dcf2e69b18870, user_id=c10745983f34485d86ae13dd09b5fa02, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:45.887663 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4de19-f79c-48b2-959f-4b486755b9b6 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:45.957713 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1268/2524] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:45 2022] PUT /identity/v3/projects/85f00d3c21e34397902dcf2e69b18870/users/c10745983f34485d86ae13dd09b5fa02/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:45.969882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521b446-335d-4496-b52e-f1de59603ab3 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:45.970175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521b446-335d-4496-b52e-f1de59603ab3 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:45.970497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521b446-335d-4496-b52e-f1de59603ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:46.000117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-4521b446-335d-4496-b52e-f1de59603ab3 None None] MFA Rules not processed for user `c10745983f34485d86ae13dd09b5fa02`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:46.001337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4521b446-335d-4496-b52e-f1de59603ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:46.067883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1257/2525] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:46.072454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4de19-f79c-48b2-959f-4b486755b9b6 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:46.137997 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4de19-f79c-48b2-959f-4b486755b9b6 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:46.143774 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92037ab2-df93-445c-83e4-f13bc83f9122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:46.144578 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92037ab2-df93-445c-83e4-f13bc83f9122 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:46.144878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92037ab2-df93-445c-83e4-f13bc83f9122 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:46.145161 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92037ab2-df93-445c-83e4-f13bc83f9122 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:46.145964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92037ab2-df93-445c-83e4-f13bc83f9122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:46.147815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92037ab2-df93-445c-83e4-f13bc83f9122 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:46.239092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1269/2526] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:46 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:16:46.243188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521b446-335d-4496-b52e-f1de59603ab3 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:46.254744 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521b446-335d-4496-b52e-f1de59603ab3 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:46.314569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d9d33-ec24-47d3-b068-a8943b4bedb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:46.315237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d9d33-ec24-47d3-b068-a8943b4bedb5 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:46.315448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d9d33-ec24-47d3-b068-a8943b4bedb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:46.315738 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d9d33-ec24-47d3-b068-a8943b4bedb5 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:46.316433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d9d33-ec24-47d3-b068-a8943b4bedb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:46.318386 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d9d33-ec24-47d3-b068-a8943b4bedb5 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:46.412369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1258/2527] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:46 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:46.416423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92037ab2-df93-445c-83e4-f13bc83f9122 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:46.428150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92037ab2-df93-445c-83e4-f13bc83f9122 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:46.432869 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be7a72-9162-4405-a7e9-35d73ce90a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:46.433544 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be7a72-9162-4405-a7e9-35d73ce90a4b admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:46.433826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be7a72-9162-4405-a7e9-35d73ce90a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:46.434103 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be7a72-9162-4405-a7e9-35d73ce90a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:46.434859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be7a72-9162-4405-a7e9-35d73ce90a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:46.436519 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be7a72-9162-4405-a7e9-35d73ce90a4b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:46.447235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1270/2528] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:46.451526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d9d33-ec24-47d3-b068-a8943b4bedb5 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:46.462739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d9d33-ec24-47d3-b068-a8943b4bedb5 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:46.468344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb443c-d941-4ace-b6cf-cc671ac45d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:46.469055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb443c-d941-4ace-b6cf-cc671ac45d3f admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:46.469368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb443c-d941-4ace-b6cf-cc671ac45d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:46.469655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb443c-d941-4ace-b6cf-cc671ac45d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:46.470421 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb443c-d941-4ace-b6cf-cc671ac45d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:46.472046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb443c-d941-4ace-b6cf-cc671ac45d3f admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:46.482706 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1259/2529] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:46.487024 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be7a72-9162-4405-a7e9-35d73ce90a4b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:46.497653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be7a72-9162-4405-a7e9-35d73ce90a4b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:46.503286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48925990-0ed2-4d6b-a072-743a7b2ea109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:46.504195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48925990-0ed2-4d6b-a072-743a7b2ea109 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:46.504481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48925990-0ed2-4d6b-a072-743a7b2ea109 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:46.504761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48925990-0ed2-4d6b-a072-743a7b2ea109 admin admin] PATH_INFO: `/v3/projects/55342a41712546e5a56f8071514d229c/users/8deadded73d644d793ebcbe5ca5c8068/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:46.521348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48925990-0ed2-4d6b-a072-743a7b2ea109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55342a41712546e5a56f8071514d229c, user_id=8deadded73d644d793ebcbe5ca5c8068, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:46.525075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48925990-0ed2-4d6b-a072-743a7b2ea109 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:46.601636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1271/2530] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:46 2022] PUT /identity/v3/projects/55342a41712546e5a56f8071514d229c/users/8deadded73d644d793ebcbe5ca5c8068/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:46.612691 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2770d6-dd53-42b1-af46-b3e54d84a1d7 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:46.612990 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2770d6-dd53-42b1-af46-b3e54d84a1d7 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:46.613305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2770d6-dd53-42b1-af46-b3e54d84a1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:46.642644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ff2770d6-dd53-42b1-af46-b3e54d84a1d7 None None] MFA Rules not processed for user `8deadded73d644d793ebcbe5ca5c8068`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:46.643865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ff2770d6-dd53-42b1-af46-b3e54d84a1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:46.679963 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:46.710653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1260/2531] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:46.714925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48925990-0ed2-4d6b-a072-743a7b2ea109 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:46.728067 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:46.780489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48925990-0ed2-4d6b-a072-743a7b2ea109 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:46.785673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd9767-3c3c-40b2-aa13-da663c695462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:46.786375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd9767-3c3c-40b2-aa13-da663c695462 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:46.786668 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd9767-3c3c-40b2-aa13-da663c695462 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:46.786951 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd9767-3c3c-40b2-aa13-da663c695462 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:46.787654 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd9767-3c3c-40b2-aa13-da663c695462 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:46.789673 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd9767-3c3c-40b2-aa13-da663c695462 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:46.918914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1272/2532] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:16:46 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:46.923583 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2770d6-dd53-42b1-af46-b3e54d84a1d7 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:46.934696 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2770d6-dd53-42b1-af46-b3e54d84a1d7 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:47.002050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6832c-3759-4649-890a-486684383c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:47.002832 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6832c-3759-4649-890a-486684383c2c admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:47.003172 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6832c-3759-4649-890a-486684383c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:47.003458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6832c-3759-4649-890a-486684383c2c admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:47.004209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6832c-3759-4649-890a-486684383c2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:47.006178 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6832c-3759-4649-890a-486684383c2c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:47.101451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1261/2533] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:16:46 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:16:47.105516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd9767-3c3c-40b2-aa13-da663c695462 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:47.117330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd9767-3c3c-40b2-aa13-da663c695462 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:47.122530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda3a5c-e52b-4286-89c4-b56bae5f5e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:47.123207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda3a5c-e52b-4286-89c4-b56bae5f5e9a admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:47.123495 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda3a5c-e52b-4286-89c4-b56bae5f5e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:47.124799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda3a5c-e52b-4286-89c4-b56bae5f5e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:47.125781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda3a5c-e52b-4286-89c4-b56bae5f5e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:47.127454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda3a5c-e52b-4286-89c4-b56bae5f5e9a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:47.138209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1273/2534] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:47.143002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6832c-3759-4649-890a-486684383c2c admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:47.153528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6832c-3759-4649-890a-486684383c2c admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:47.158613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22983137-bdf1-4919-aedd-3f832f3331a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:47.159531 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22983137-bdf1-4919-aedd-3f832f3331a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:47.159891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22983137-bdf1-4919-aedd-3f832f3331a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:47.160184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22983137-bdf1-4919-aedd-3f832f3331a1 admin admin] PATH_INFO: `/v3/projects/037bf376d5dd4b53ab1bac6cd552d064/users/9f8f65474e894f7b8e95b2bf22ec25fc/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:47.176624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22983137-bdf1-4919-aedd-3f832f3331a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=037bf376d5dd4b53ab1bac6cd552d064, user_id=9f8f65474e894f7b8e95b2bf22ec25fc, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:47.179261 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22983137-bdf1-4919-aedd-3f832f3331a1 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:47.269086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1262/2535] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:47 2022] PUT /identity/v3/projects/037bf376d5dd4b53ab1bac6cd552d064/users/9f8f65474e894f7b8e95b2bf22ec25fc/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:47.277038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda3a5c-e52b-4286-89c4-b56bae5f5e9a admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:47.288619 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda3a5c-e52b-4286-89c4-b56bae5f5e9a admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:47.372694 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1456ea2-1be0-4973-8053-2a32b928fd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:47.373501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1456ea2-1be0-4973-8053-2a32b928fd96 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:47.374031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1456ea2-1be0-4973-8053-2a32b928fd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:47.374413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1456ea2-1be0-4973-8053-2a32b928fd96 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:47.375628 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1456ea2-1be0-4973-8053-2a32b928fd96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:47.377261 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1456ea2-1be0-4973-8053-2a32b928fd96 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:47.391042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1274/2536] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:16:47.395813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22983137-bdf1-4919-aedd-3f832f3331a1 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:47.407946 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22983137-bdf1-4919-aedd-3f832f3331a1 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:47.413182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9286d5-3ad7-4055-a8c6-90aeda4321bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:47.414179 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9286d5-3ad7-4055-a8c6-90aeda4321bc admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:47.415033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9286d5-3ad7-4055-a8c6-90aeda4321bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:47.415416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9286d5-3ad7-4055-a8c6-90aeda4321bc admin admin] PATH_INFO: `/v3/projects/037bf376d5dd4b53ab1bac6cd552d064/users/9f8f65474e894f7b8e95b2bf22ec25fc/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:47.417608 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9286d5-3ad7-4055-a8c6-90aeda4321bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=037bf376d5dd4b53ab1bac6cd552d064, user_id=9f8f65474e894f7b8e95b2bf22ec25fc, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:47.420345 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9286d5-3ad7-4055-a8c6-90aeda4321bc admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:47.432856 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1263/2537] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:16:47 2022] PUT /identity/v3/projects/037bf376d5dd4b53ab1bac6cd552d064/users/9f8f65474e894f7b8e95b2bf22ec25fc/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:47.443268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f80d4-bbb6-494d-92cd-e25da06c8ad2 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:47.443559 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f80d4-bbb6-494d-92cd-e25da06c8ad2 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:47.443844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f80d4-bbb6-494d-92cd-e25da06c8ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:47.475448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d89f80d4-bbb6-494d-92cd-e25da06c8ad2 None None] MFA Rules not processed for user `9f8f65474e894f7b8e95b2bf22ec25fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:47.476761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d89f80d4-bbb6-494d-92cd-e25da06c8ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:47.576859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1275/2538] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:47.594295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9286d5-3ad7-4055-a8c6-90aeda4321bc admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:47.596909 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ff9286d5-3ad7-4055-a8c6-90aeda4321bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:47.668203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9286d5-3ad7-4055-a8c6-90aeda4321bc admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:47.672242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30266ea-b604-4b26-b531-d59acf90eb47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:47.674251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30266ea-b604-4b26-b531-d59acf90eb47 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:47.674251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30266ea-b604-4b26-b531-d59acf90eb47 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:47.674251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30266ea-b604-4b26-b531-d59acf90eb47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:47.728263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:47.747548 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30266ea-b604-4b26-b531-d59acf90eb47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:47.749437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30266ea-b604-4b26-b531-d59acf90eb47 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:47.760594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1264/2539] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:50.928454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f80d4-bbb6-494d-92cd-e25da06c8ad2 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:50.939149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f80d4-bbb6-494d-92cd-e25da06c8ad2 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:50.943018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c8e86-2804-4fea-9835-2ddd5f473d27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:50.943741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c8e86-2804-4fea-9835-2ddd5f473d27 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:50.944089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c8e86-2804-4fea-9835-2ddd5f473d27 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:50.944477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c8e86-2804-4fea-9835-2ddd5f473d27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:50.949158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:51.010592 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c8e86-2804-4fea-9835-2ddd5f473d27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:51.012143 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c8e86-2804-4fea-9835-2ddd5f473d27 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:51.027848 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1276/2540] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:52.642105 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30266ea-b604-4b26-b531-d59acf90eb47 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:52.651012 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30266ea-b604-4b26-b531-d59acf90eb47 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:52.655327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d0e2d-f76f-4212-a0c0-1bebcdcc869d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:52.655982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d0e2d-f76f-4212-a0c0-1bebcdcc869d service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:52.656294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d0e2d-f76f-4212-a0c0-1bebcdcc869d service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:52.656573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d0e2d-f76f-4212-a0c0-1bebcdcc869d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:52.667011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d0e2d-f76f-4212-a0c0-1bebcdcc869d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:52.668501 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d0e2d-f76f-4212-a0c0-1bebcdcc869d service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:52.679441 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1265/2541] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:58.413704 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c8e86-2804-4fea-9835-2ddd5f473d27 service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:58.481494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c8e86-2804-4fea-9835-2ddd5f473d27 service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:58.487172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:58.488056 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:58.488344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:58.488620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] PATH_INFO: `/v3/users/c10745983f34485d86ae13dd09b5fa02` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:58.489834 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] RBAC: Authorizing `identity:delete_user(user_id=c10745983f34485d86ae13dd09b5fa02)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:58.491797 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:58.674297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c10745983f34485d86ae13dd09b5fa02'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:58.675023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c10745983f34485d86ae13dd09b5fa02'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:58.682308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c10745983f34485d86ae13dd09b5fa02'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:58.694855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c10745983f34485d86ae13dd09b5fa02'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:58.762798 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c10745983f34485d86ae13dd09b5fa02'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:58.769626 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1277/2542] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:58 2022] DELETE /identity/v3/users/c10745983f34485d86ae13dd09b5fa02 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:58.774247 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9d16c-32f8-4304-88d1-6429ddeef53a None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:58.774693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9d16c-32f8-4304-88d1-6429ddeef53a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:58.774855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9d16c-32f8-4304-88d1-6429ddeef53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:58.794858 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-2dc9d16c-32f8-4304-88d1-6429ddeef53a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:58.796736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2dc9d16c-32f8-4304-88d1-6429ddeef53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:58.858098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1266/2543] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:58.867902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:58.869629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:58.899382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:58.930248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b7887-e1c1-4533-9a76-d5c5fb06caed admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:58.934061 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dffe7-af0a-4140-84ee-e6575b065d1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:58.934751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dffe7-af0a-4140-84ee-e6575b065d1e service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:58.935048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dffe7-af0a-4140-84ee-e6575b065d1e service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:58.935330 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dffe7-af0a-4140-84ee-e6575b065d1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:58.945888 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46dffe7-af0a-4140-84ee-e6575b065d1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:58.947345 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46dffe7-af0a-4140-84ee-e6575b065d1e service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:58.958030 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1278/2544] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:59.507449 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9d16c-32f8-4304-88d1-6429ddeef53a None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:59.509996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2dc9d16c-32f8-4304-88d1-6429ddeef53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:59.521408 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9d16c-32f8-4304-88d1-6429ddeef53a None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:59.526572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:59.527467 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:59.527761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:59.528046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] PATH_INFO: `/v3/projects/85f00d3c21e34397902dcf2e69b18870` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:59.530010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85f00d3c21e34397902dcf2e69b18870)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:59.532017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:59.660347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85f00d3c21e34397902dcf2e69b18870', 'request_id': 'req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:59.745860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85f00d3c21e34397902dcf2e69b18870', 'request_id': 'req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:59.757129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85f00d3c21e34397902dcf2e69b18870', 'request_id': 'req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:59.775847 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] The token cache is being invalidate because project 85f00d3c21e34397902dcf2e69b18870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:16:59.776492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:59.776868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:16:59.778584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1267/2545] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:16:59 2022] DELETE /identity/v3/projects/85f00d3c21e34397902dcf2e69b18870 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:16:59.784866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dffe7-af0a-4140-84ee-e6575b065d1e service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:59.794939 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dffe7-af0a-4140-84ee-e6575b065d1e service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:59.853497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:59.854827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:59.854827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:59.855079 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] PATH_INFO: `/v3/users/8deadded73d644d793ebcbe5ca5c8068` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:59.856319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8deadded73d644d793ebcbe5ca5c8068)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:59.858730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:00.041594 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8deadded73d644d793ebcbe5ca5c8068'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:00.042225 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8deadded73d644d793ebcbe5ca5c8068'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:00.050958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8deadded73d644d793ebcbe5ca5c8068'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:00.063545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8deadded73d644d793ebcbe5ca5c8068'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:00.145282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8deadded73d644d793ebcbe5ca5c8068'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:00.151782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1279/2546] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:16:59 2022] DELETE /identity/v3/users/8deadded73d644d793ebcbe5ca5c8068 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:17:00.586236 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:00.587778 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:00.602437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b485c2-bde3-43f6-9a30-6fcd2a1a4ba7 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:00.655217 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:00.664343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:00.665223 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:00.665527 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:00.665837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] PATH_INFO: `/v3/projects/55342a41712546e5a56f8071514d229c` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:00.667064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55342a41712546e5a56f8071514d229c)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:00.669009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:00.787924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55342a41712546e5a56f8071514d229c', 'request_id': 'req-d28fd76d-e601-4697-82c4-5669088ed6f5'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:00.905949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55342a41712546e5a56f8071514d229c', 'request_id': 'req-d28fd76d-e601-4697-82c4-5669088ed6f5'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:00.912746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55342a41712546e5a56f8071514d229c', 'request_id': 'req-d28fd76d-e601-4697-82c4-5669088ed6f5'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:00.926588 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] The token cache is being invalidate because project 55342a41712546e5a56f8071514d229c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:17:00.927006 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:00.927755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:00.929796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1268/2547] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:17:00 2022] DELETE /identity/v3/projects/55342a41712546e5a56f8071514d229c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:17:00.933664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:00.935175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:00.952322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a12642-d99c-4082-ad4e-0e69d875f29e admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:01.000736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:01.013314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:01.015055 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:01.015511 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:01.015790 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] PATH_INFO: `/v3/users/9f8f65474e894f7b8e95b2bf22ec25fc` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:01.017004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f8f65474e894f7b8e95b2bf22ec25fc)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:01.018971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:01.253133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f8f65474e894f7b8e95b2bf22ec25fc'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:01.253859 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f8f65474e894f7b8e95b2bf22ec25fc'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:01.262530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f8f65474e894f7b8e95b2bf22ec25fc'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:01.275470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f8f65474e894f7b8e95b2bf22ec25fc'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:01.374408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f8f65474e894f7b8e95b2bf22ec25fc'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:01.381079 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1280/2548] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:17:00 2022] DELETE /identity/v3/users/9f8f65474e894f7b8e95b2bf22ec25fc => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:17:01.884625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:01.886101 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:01.907715 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fd76d-e601-4697-82c4-5669088ed6f5 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:01.970988 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:01.971921 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:01.972215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:01.972540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9 admin admin] PATH_INFO: `/v3/projects/037bf376d5dd4b53ab1bac6cd552d064` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:01.973740 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=037bf376d5dd4b53ab1bac6cd552d064)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:01.975738 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:02.068053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '037bf376d5dd4b53ab1bac6cd552d064', 'request_id': 'req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:02.129916 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '037bf376d5dd4b53ab1bac6cd552d064', 'request_id': 'req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:02.140813 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '037bf376d5dd4b53ab1bac6cd552d064', 'request_id': 'req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:02.152766 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9 admin admin] The token cache is being invalidate because project 037bf376d5dd4b53ab1bac6cd552d064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:17:02.153061 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:02.153718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-4fa4ed85-e9f0-4bef-9ddc-85b5dd758fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:17:02.156746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1269/2549] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:17:01 2022] DELETE /identity/v3/projects/037bf376d5dd4b53ab1bac6cd552d064 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:17:08.190804 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:08.193442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:08.273228 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b74e6-94f7-4c74-b58f-6534a16ba5f1 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:08.331343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:08.340288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e093384-a9e4-423f-827d-137880c949f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:08.342392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e093384-a9e4-423f-827d-137880c949f6 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:08.342392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e093384-a9e4-423f-827d-137880c949f6 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:08.342392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e093384-a9e4-423f-827d-137880c949f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:08.344404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8e093384-a9e4-423f-827d-137880c949f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:08.406763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e093384-a9e4-423f-827d-137880c949f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:08.408544 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e093384-a9e4-423f-827d-137880c949f6 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:08.418469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1281/2550] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:12.149692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=78,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:18.454069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=27 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:38.300773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4597ce-1a67-4cd3-b8f1-f826c94a4f7f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:38.301288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4597ce-1a67-4cd3-b8f1-f826c94a4f7f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:38.301379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4597ce-1a67-4cd3-b8f1-f826c94a4f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:38.355134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ab4597ce-1a67-4cd3-b8f1-f826c94a4f7f None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:38.356303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ab4597ce-1a67-4cd3-b8f1-f826c94a4f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:38.429442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1270/2551] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:17:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:17:38.585220 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e093384-a9e4-423f-827d-137880c949f6 service nova] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:38.586717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8e093384-a9e4-423f-827d-137880c949f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:38.679843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e093384-a9e4-423f-827d-137880c949f6 service nova] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:38.685648 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023494d-1692-494d-b600-7af780b4dddb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:38.686410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023494d-1692-494d-b600-7af780b4dddb service swift] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:38.686701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023494d-1692-494d-b600-7af780b4dddb service swift] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:38.686983 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023494d-1692-494d-b600-7af780b4dddb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:38.695883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023494d-1692-494d-b600-7af780b4dddb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:38.697469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023494d-1692-494d-b600-7af780b4dddb service swift] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:38.705717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1282/2552] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:17:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:17:48.424507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=40 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:48.702410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=41 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:55.254176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4597ce-1a67-4cd3-b8f1-f826c94a4f7f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:55.256042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ab4597ce-1a67-4cd3-b8f1-f826c94a4f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:55.332760 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4597ce-1a67-4cd3-b8f1-f826c94a4f7f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:55.337706 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384c94b-47bb-40ec-9472-dbb788bac2f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:55.338406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384c94b-47bb-40ec-9472-dbb788bac2f6 service placement] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:55.338691 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384c94b-47bb-40ec-9472-dbb788bac2f6 service placement] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:55.349390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384c94b-47bb-40ec-9472-dbb788bac2f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:55.349390 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2384c94b-47bb-40ec-9472-dbb788bac2f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:55.353881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384c94b-47bb-40ec-9472-dbb788bac2f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:55.355529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384c94b-47bb-40ec-9472-dbb788bac2f6 service placement] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:55.366636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1271/2553] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:59.151916 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abfc33-a1bd-4819-80e3-b92206c3da60 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:59.152473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abfc33-a1bd-4819-80e3-b92206c3da60 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:59.152473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abfc33-a1bd-4819-80e3-b92206c3da60 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:59.154566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1283/2554] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 19:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:17:59.160344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89627b5e-c8e7-47bc-bf9d-ac667a923bdd None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:59.160889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89627b5e-c8e7-47bc-bf9d-ac667a923bdd None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:59.160889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89627b5e-c8e7-47bc-bf9d-ac667a923bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:59.202043 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-89627b5e-c8e7-47bc-bf9d-ac667a923bdd None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:59.203983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-89627b5e-c8e7-47bc-bf9d-ac667a923bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:59.214311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1272/2555] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 19:17:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:17:59.226742 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abfc33-a1bd-4819-80e3-b92206c3da60 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:59.228457 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-36abfc33-a1bd-4819-80e3-b92206c3da60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:59.289185 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abfc33-a1bd-4819-80e3-b92206c3da60 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:59.292660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd57515-46e1-4c6c-98c8-a857e7e9024d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28954d67689a46ee975fdfc9139e41eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:59.293352 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd57515-46e1-4c6c-98c8-a857e7e9024d service glance] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:59.294179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd57515-46e1-4c6c-98c8-a857e7e9024d service glance] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:59.294593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd57515-46e1-4c6c-98c8-a857e7e9024d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:59.305095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd57515-46e1-4c6c-98c8-a857e7e9024d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:59.308996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd57515-46e1-4c6c-98c8-a857e7e9024d service glance] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:59.318012 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1284/2556] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:59.551052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc2537-0716-4beb-8d99-b1fc70921a36 None None] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:59.551052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc2537-0716-4beb-8d99-b1fc70921a36 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:59.551052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc2537-0716-4beb-8d99-b1fc70921a36 None None] PATH_INFO: `/` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:59.551052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1273/2557] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 19:17:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:17:59.553709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31570b5f-3844-45db-a865-88690114c13d None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:59.554007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31570b5f-3844-45db-a865-88690114c13d None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:59.554292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31570b5f-3844-45db-a865-88690114c13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:59.592843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-31570b5f-3844-45db-a865-88690114c13d None None] MFA Rules not processed for user `ffcf280bc6d54b2fb2d22634468ac6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:59.594058 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-31570b5f-3844-45db-a865-88690114c13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:59.647529 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1285/2558] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 19:17:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:17:59.659953 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc2537-0716-4beb-8d99-b1fc70921a36 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:59.661679 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-69cc2537-0716-4beb-8d99-b1fc70921a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:59.676167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc2537-0716-4beb-8d99-b1fc70921a36 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:59.679675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ef79c-47f3-4bba-871d-a4caa599e9cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:59.680318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ef79c-47f3-4bba-871d-a4caa599e9cb service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:59.680602 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ef79c-47f3-4bba-871d-a4caa599e9cb service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:59.680879 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ef79c-47f3-4bba-871d-a4caa599e9cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:59.689304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ef79c-47f3-4bba-871d-a4caa599e9cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:59.690778 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ef79c-47f3-4bba-871d-a4caa599e9cb service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:59.698950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1274/2559] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:59.785305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dca66-6594-4a93-8822-7e6974ff1d86 None None] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:59.785595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dca66-6594-4a93-8822-7e6974ff1d86 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:59.785876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dca66-6594-4a93-8822-7e6974ff1d86 None None] PATH_INFO: `/` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:59.787377 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1286/2560] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 19:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:17:59.793985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e9113-83ba-420a-9c5b-4d531238fb05 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:59.794657 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e9113-83ba-420a-9c5b-4d531238fb05 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:59.795016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e9113-83ba-420a-9c5b-4d531238fb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:59.819956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b28e9113-83ba-420a-9c5b-4d531238fb05 None None] MFA Rules not processed for user `e628cb75e0c74f27a390945c8a04335c`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:59.821163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b28e9113-83ba-420a-9c5b-4d531238fb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:59.833127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1275/2561] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 19:17:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:17:59.846790 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534dca66-6594-4a93-8822-7e6974ff1d86 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:59.856822 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534dca66-6594-4a93-8822-7e6974ff1d86 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:59.860561 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8beaa-fc95-4987-a929-d6af1b66ff3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:59.861205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8beaa-fc95-4987-a929-d6af1b66ff3c service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:59.861489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8beaa-fc95-4987-a929-d6af1b66ff3c service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:59.861776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8beaa-fc95-4987-a929-d6af1b66ff3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:59.873446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8beaa-fc95-4987-a929-d6af1b66ff3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:59.874953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8beaa-fc95-4987-a929-d6af1b66ff3c service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:59.886371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1287/2562] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:18:02.337532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d73f54-2b84-4f13-b862-b34f6325a0ee None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:02.338045 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d73f54-2b84-4f13-b862-b34f6325a0ee None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:02.338156 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d73f54-2b84-4f13-b862-b34f6325a0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:02.357837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-41d73f54-2b84-4f13-b862-b34f6325a0ee None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:18:02.359112 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-41d73f54-2b84-4f13-b862-b34f6325a0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:02.366654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1276/2563] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:18:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:18:02.379669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8beaa-fc95-4987-a929-d6af1b66ff3c service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:18:02.392120 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8beaa-fc95-4987-a929-d6af1b66ff3c service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:18:02.397046 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49752180-27f9-4c8f-aaa8-b8c32f7fb293 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:18:02.397683 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49752180-27f9-4c8f-aaa8-b8c32f7fb293 service swift] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:02.397973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49752180-27f9-4c8f-aaa8-b8c32f7fb293 service swift] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:02.398406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49752180-27f9-4c8f-aaa8-b8c32f7fb293 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:02.407370 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49752180-27f9-4c8f-aaa8-b8c32f7fb293 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:18:02.409010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49752180-27f9-4c8f-aaa8-b8c32f7fb293 service swift] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:18:02.417672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1288/2564] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:18:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 19:18:12.353112 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=53 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:12.414466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=67 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:23.321414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b1af4-298d-4780-876b-565b4236d5a7 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:23.321887 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b1af4-298d-4780-876b-565b4236d5a7 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:23.321994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b1af4-298d-4780-876b-565b4236d5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:23.341411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-ca1b1af4-298d-4780-876b-565b4236d5a7 None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:18:23.342683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ca1b1af4-298d-4780-876b-565b4236d5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:23.350121 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1277/2565] 10.222.0.17 () {58 vars in 967 bytes} [Mon Jul 25 19:18:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:18:23.363452 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49752180-27f9-4c8f-aaa8-b8c32f7fb293 service swift] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:18:23.374980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49752180-27f9-4c8f-aaa8-b8c32f7fb293 service swift] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:18:23.380318 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1a9b6-2f26-48bf-a14b-36536e71b712 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:18:23.380963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1a9b6-2f26-48bf-a14b-36536e71b712 service swift] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:23.381247 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1a9b6-2f26-48bf-a14b-36536e71b712 service swift] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:23.381527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1a9b6-2f26-48bf-a14b-36536e71b712 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:23.390159 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1a9b6-2f26-48bf-a14b-36536e71b712 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:18:23.391769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1a9b6-2f26-48bf-a14b-36536e71b712 service swift] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:18:23.399785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1289/2566] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:18:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:18:33.336732 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:33.396562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=6 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:49.713169 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b1af4-298d-4780-876b-565b4236d5a7 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:18:49.718309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ca1b1af4-298d-4780-876b-565b4236d5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:49.734426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b1af4-298d-4780-876b-565b4236d5a7 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:18:49.741885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa9f22-f0b0-49a1-addd-9961a50f0209 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:18:49.743124 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa9f22-f0b0-49a1-addd-9961a50f0209 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:49.743511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa9f22-f0b0-49a1-addd-9961a50f0209 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:49.743912 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa9f22-f0b0-49a1-addd-9961a50f0209 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:49.745984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1baa9f22-f0b0-49a1-addd-9961a50f0209 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:49.763532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa9f22-f0b0-49a1-addd-9961a50f0209 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:18:49.765765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa9f22-f0b0-49a1-addd-9961a50f0209 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:18:49.782349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1278/2567] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:18:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:18:56.561850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1a9b6-2f26-48bf-a14b-36536e71b712 service swift] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:18:56.561850 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1a9b6-2f26-48bf-a14b-36536e71b712 service swift] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:18:56.562429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ae058-6215-4323-95ea-72c00d1a2b6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:18:56.563071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ae058-6215-4323-95ea-72c00d1a2b6d service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:56.563361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ae058-6215-4323-95ea-72c00d1a2b6d service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:56.563670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ae058-6215-4323-95ea-72c00d1a2b6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:56.565940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fe0ae058-6215-4323-95ea-72c00d1a2b6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:56.631254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ae058-6215-4323-95ea-72c00d1a2b6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:18:56.632896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ae058-6215-4323-95ea-72c00d1a2b6d service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:18:56.646526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1290/2568] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:18:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:18:59.775050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=6 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:00.321674 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa9f22-f0b0-49a1-addd-9961a50f0209 service nova] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:00.323174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1baa9f22-f0b0-49a1-addd-9961a50f0209 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:00.338402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa9f22-f0b0-49a1-addd-9961a50f0209 service nova] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:00.344220 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec535da-90fe-4faa-9054-c82b8326c7c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:00.344898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec535da-90fe-4faa-9054-c82b8326c7c4 service placement] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:00.345219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec535da-90fe-4faa-9054-c82b8326c7c4 service placement] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:00.345556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec535da-90fe-4faa-9054-c82b8326c7c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:00.348090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0ec535da-90fe-4faa-9054-c82b8326c7c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:00.361095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec535da-90fe-4faa-9054-c82b8326c7c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:00.363107 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec535da-90fe-4faa-9054-c82b8326c7c4 service placement] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:00.375004 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1279/2569] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:19:00.459558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ae058-6215-4323-95ea-72c00d1a2b6d service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:00.460144 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-fe0ae058-6215-4323-95ea-72c00d1a2b6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:00.473421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ae058-6215-4323-95ea-72c00d1a2b6d service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:00.478072 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:00.478997 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:00.479286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:00.479564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] PATH_INFO: `/v3/users/592de777b94148b791e18bf898386f34` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:00.480784 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=592de777b94148b791e18bf898386f34)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:00.482741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:00.672279 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '592de777b94148b791e18bf898386f34'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:00.673048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '592de777b94148b791e18bf898386f34'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:00.680126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '592de777b94148b791e18bf898386f34'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:00.692447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '592de777b94148b791e18bf898386f34'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:00.760147 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '592de777b94148b791e18bf898386f34'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:00.766940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1291/2570] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:19:00 2022] DELETE /identity/v3/users/592de777b94148b791e18bf898386f34 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:19:00.988272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec535da-90fe-4faa-9054-c82b8326c7c4 service placement] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:00.999138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec535da-90fe-4faa-9054-c82b8326c7c4 service placement] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:01.052388 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f427d-ef35-4d44-94e0-777db12200b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:01.053285 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f427d-ef35-4d44-94e0-777db12200b4 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:01.053415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f427d-ef35-4d44-94e0-777db12200b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:01.053653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f427d-ef35-4d44-94e0-777db12200b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:01.054268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:01.055581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e94f427d-ef35-4d44-94e0-777db12200b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:01.055949 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:01.136127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f427d-ef35-4d44-94e0-777db12200b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:01.137607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f427d-ef35-4d44-94e0-777db12200b4 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:01.138526 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6101ff-a6ea-4cb5-ba3b-fdf4495d2c17 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:01.143744 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:01.145310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:01.145310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:01.145310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db admin admin] PATH_INFO: `/v3/projects/5e929654eb2b4c1ca969174e94aa8de2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:01.146594 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e929654eb2b4c1ca969174e94aa8de2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:01.147569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1280/2571] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:19:01.148694 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:01.161084 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:01.277429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e929654eb2b4c1ca969174e94aa8de2', 'request_id': 'req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:01.285046 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e929654eb2b4c1ca969174e94aa8de2', 'request_id': 'req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:01.290973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e929654eb2b4c1ca969174e94aa8de2', 'request_id': 'req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:01.360551 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db admin admin] The token cache is being invalidate because project 5e929654eb2b4c1ca969174e94aa8de2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:19:01.360847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:01.362159 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-35dee2e9-9df1-4e87-9f97-f1b41ef7c3db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:01.364035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1292/2572] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:19:01 2022] DELETE /identity/v3/projects/5e929654eb2b4c1ca969174e94aa8de2 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:19:01.369460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fd647-3291-49b8-b1ef-b4c30db96363 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:01.369801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fd647-3291-49b8-b1ef-b4c30db96363 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:01.370268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fd647-3291-49b8-b1ef-b4c30db96363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:01.391174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-750fd647-3291-49b8-b1ef-b4c30db96363 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:01.392454 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-750fd647-3291-49b8-b1ef-b4c30db96363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:01.459287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1281/2573] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:01.468362 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61395a-e6fb-4f93-82ca-01d1dc1785a3 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:01.468902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61395a-e6fb-4f93-82ca-01d1dc1785a3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:01.468902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61395a-e6fb-4f93-82ca-01d1dc1785a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:01.492274 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-cd61395a-e6fb-4f93-82ca-01d1dc1785a3 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:01.493518 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-cd61395a-e6fb-4f93-82ca-01d1dc1785a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:01.502460 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1293/2574] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:01.506900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750fd647-3291-49b8-b1ef-b4c30db96363 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:01.518896 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750fd647-3291-49b8-b1ef-b4c30db96363 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:01.523896 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00db46-7b9b-4689-bab7-44c8d41044b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:01.524561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00db46-7b9b-4689-bab7-44c8d41044b4 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:01.526476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00db46-7b9b-4689-bab7-44c8d41044b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:01.526476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00db46-7b9b-4689-bab7-44c8d41044b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:01.526476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00db46-7b9b-4689-bab7-44c8d41044b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:19:01.526476 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00db46-7b9b-4689-bab7-44c8d41044b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:01.527814 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00db46-7b9b-4689-bab7-44c8d41044b4 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:01.540950 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1282/2575] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:19:01.545892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61395a-e6fb-4f93-82ca-01d1dc1785a3 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:01.557875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61395a-e6fb-4f93-82ca-01d1dc1785a3 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:01.563028 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee38a0-6980-4f6b-908d-3cbce5569940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:01.563705 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee38a0-6980-4f6b-908d-3cbce5569940 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:01.563993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee38a0-6980-4f6b-908d-3cbce5569940 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:01.564269 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee38a0-6980-4f6b-908d-3cbce5569940 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:01.565124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee38a0-6980-4f6b-908d-3cbce5569940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:01.567073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee38a0-6980-4f6b-908d-3cbce5569940 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:01.666783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1294/2576] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:19:01 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:19:01.672549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00db46-7b9b-4689-bab7-44c8d41044b4 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:01.683875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00db46-7b9b-4689-bab7-44c8d41044b4 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:01.716915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:01.754516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154dc026-0f87-4251-8a9a-56dc2a6a17e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:01.755219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154dc026-0f87-4251-8a9a-56dc2a6a17e9 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:01.755516 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154dc026-0f87-4251-8a9a-56dc2a6a17e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:01.755791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154dc026-0f87-4251-8a9a-56dc2a6a17e9 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:01.756495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154dc026-0f87-4251-8a9a-56dc2a6a17e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:01.758529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154dc026-0f87-4251-8a9a-56dc2a6a17e9 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:01.858575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1283/2577] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:19:01 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:19:01.862606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee38a0-6980-4f6b-908d-3cbce5569940 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:01.873339 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee38a0-6980-4f6b-908d-3cbce5569940 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:01.878481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685166a6-e0db-4b5c-b804-af3d8a7d832b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:01.879164 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685166a6-e0db-4b5c-b804-af3d8a7d832b admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:01.879451 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685166a6-e0db-4b5c-b804-af3d8a7d832b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:01.879756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685166a6-e0db-4b5c-b804-af3d8a7d832b admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:01.880621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685166a6-e0db-4b5c-b804-af3d8a7d832b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:01.882248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685166a6-e0db-4b5c-b804-af3d8a7d832b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:01.895308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1295/2578] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:01.903165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154dc026-0f87-4251-8a9a-56dc2a6a17e9 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:01.914395 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154dc026-0f87-4251-8a9a-56dc2a6a17e9 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:01.919276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7bcde-d7a9-434b-94da-a34976dbff79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:01.919966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7bcde-d7a9-434b-94da-a34976dbff79 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:01.920256 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7bcde-d7a9-434b-94da-a34976dbff79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:01.920569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7bcde-d7a9-434b-94da-a34976dbff79 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:01.921311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7bcde-d7a9-434b-94da-a34976dbff79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:01.922969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7bcde-d7a9-434b-94da-a34976dbff79 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:01.935181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1284/2579] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:01.939449 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685166a6-e0db-4b5c-b804-af3d8a7d832b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:01.950409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685166a6-e0db-4b5c-b804-af3d8a7d832b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:01.961256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073a3e0-3b4c-4b12-b9d0-278d8abbfdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:01.961256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073a3e0-3b4c-4b12-b9d0-278d8abbfdf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:01.961256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073a3e0-3b4c-4b12-b9d0-278d8abbfdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:01.961256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073a3e0-3b4c-4b12-b9d0-278d8abbfdf6 admin admin] PATH_INFO: `/v3/projects/c283d74bc1404c078fa9c350c8acfe4a/users/61d6214556954b778dc86a0d376a8d6e/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:01.973811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073a3e0-3b4c-4b12-b9d0-278d8abbfdf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c283d74bc1404c078fa9c350c8acfe4a, user_id=61d6214556954b778dc86a0d376a8d6e, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:01.976556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073a3e0-3b4c-4b12-b9d0-278d8abbfdf6 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:02.046048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1296/2580] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:19:01 2022] PUT /identity/v3/projects/c283d74bc1404c078fa9c350c8acfe4a/users/61d6214556954b778dc86a0d376a8d6e/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:02.051181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfb36b-ec4d-4227-aa39-ff8069b4f971 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:02.051648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfb36b-ec4d-4227-aa39-ff8069b4f971 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:02.051763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfb36b-ec4d-4227-aa39-ff8069b4f971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:02.071827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3bbfb36b-ec4d-4227-aa39-ff8069b4f971 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:02.073069 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3bbfb36b-ec4d-4227-aa39-ff8069b4f971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:02.154370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1285/2581] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:02.163774 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073a3e0-3b4c-4b12-b9d0-278d8abbfdf6 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:02.165741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3073a3e0-3b4c-4b12-b9d0-278d8abbfdf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:02.273131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073a3e0-3b4c-4b12-b9d0-278d8abbfdf6 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:02.277560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48f60b-970b-49fc-a623-36b12fcb7384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:02.278583 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48f60b-970b-49fc-a623-36b12fcb7384 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:02.279013 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48f60b-970b-49fc-a623-36b12fcb7384 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:02.279364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48f60b-970b-49fc-a623-36b12fcb7384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:02.295021 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48f60b-970b-49fc-a623-36b12fcb7384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:02.296542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48f60b-970b-49fc-a623-36b12fcb7384 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:02.310922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1297/2582] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:19:08.031094 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfb36b-ec4d-4227-aa39-ff8069b4f971 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:08.032681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3bbfb36b-ec4d-4227-aa39-ff8069b4f971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:08.052879 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfb36b-ec4d-4227-aa39-ff8069b4f971 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:08.063015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c82ff-fbf9-46cc-9924-81c16799ac95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:08.063015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c82ff-fbf9-46cc-9924-81c16799ac95 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:08.063015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c82ff-fbf9-46cc-9924-81c16799ac95 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:08.063015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c82ff-fbf9-46cc-9924-81c16799ac95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:08.063571 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-df5c82ff-fbf9-46cc-9924-81c16799ac95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:08.076076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c82ff-fbf9-46cc-9924-81c16799ac95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:08.077787 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c82ff-fbf9-46cc-9924-81c16799ac95 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:08.090473 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1286/2583] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:19:08.191409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12781fd-7985-4deb-bfa5-6b9a0322dd76 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:08.191972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12781fd-7985-4deb-bfa5-6b9a0322dd76 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:08.191972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12781fd-7985-4deb-bfa5-6b9a0322dd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:08.221661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-e12781fd-7985-4deb-bfa5-6b9a0322dd76 None None] MFA Rules not processed for user `61d6214556954b778dc86a0d376a8d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:08.222916 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e12781fd-7985-4deb-bfa5-6b9a0322dd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:08.248982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:08.292423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1298/2584] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:08.306534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c82ff-fbf9-46cc-9924-81c16799ac95 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:08.308036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-df5c82ff-fbf9-46cc-9924-81c16799ac95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:08.405736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c82ff-fbf9-46cc-9924-81c16799ac95 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:08.413685 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf8c5e-ca4a-4eac-89ec-9b2167296fa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:08.414407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf8c5e-ca4a-4eac-89ec-9b2167296fa9 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:08.414698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf8c5e-ca4a-4eac-89ec-9b2167296fa9 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:08.414977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf8c5e-ca4a-4eac-89ec-9b2167296fa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:08.425370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf8c5e-ca4a-4eac-89ec-9b2167296fa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:08.427039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf8c5e-ca4a-4eac-89ec-9b2167296fa9 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:08.436895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1287/2585] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:19:11.284593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c18f0-5ea1-4b63-a2ff-d628040baac1 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:11.285156 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c18f0-5ea1-4b63-a2ff-d628040baac1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:11.285278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c18f0-5ea1-4b63-a2ff-d628040baac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:11.332438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-649c18f0-5ea1-4b63-a2ff-d628040baac1 None None] MFA Rules not processed for user `61d6214556954b778dc86a0d376a8d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:11.335792 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-649c18f0-5ea1-4b63-a2ff-d628040baac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:11.372793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1299/2586] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:12.039781 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf8c5e-ca4a-4eac-89ec-9b2167296fa9 service nova] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:12.056624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf8c5e-ca4a-4eac-89ec-9b2167296fa9 service nova] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:12.060895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965f7e2-cb5d-4693-9bc8-7fb48e2b56f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:12.061833 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965f7e2-cb5d-4693-9bc8-7fb48e2b56f5 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:12.062126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965f7e2-cb5d-4693-9bc8-7fb48e2b56f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:12.062590 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965f7e2-cb5d-4693-9bc8-7fb48e2b56f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:12.065231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3965f7e2-cb5d-4693-9bc8-7fb48e2b56f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:12.081676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965f7e2-cb5d-4693-9bc8-7fb48e2b56f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:12.081676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965f7e2-cb5d-4693-9bc8-7fb48e2b56f5 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:12.089249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1288/2587] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:19:12.340341 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c18f0-5ea1-4b63-a2ff-d628040baac1 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:12.341652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-649c18f0-5ea1-4b63-a2ff-d628040baac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:12.430099 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c18f0-5ea1-4b63-a2ff-d628040baac1 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:12.436644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6ea12-77a4-4864-9737-7bd1f327c714 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:12.437479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ea12-77a4-4864-9737-7bd1f327c714 service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:12.437842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ea12-77a4-4864-9737-7bd1f327c714 service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:12.438195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ea12-77a4-4864-9737-7bd1f327c714 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:12.441148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-20a6ea12-77a4-4864-9737-7bd1f327c714 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:12.468036 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6ea12-77a4-4864-9737-7bd1f327c714 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:12.472151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6ea12-77a4-4864-9737-7bd1f327c714 service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:12.484555 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1300/2588] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:19:16.361027 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965f7e2-cb5d-4693-9bc8-7fb48e2b56f5 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:16.363690 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3965f7e2-cb5d-4693-9bc8-7fb48e2b56f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:16.386164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965f7e2-cb5d-4693-9bc8-7fb48e2b56f5 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:16.391257 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:16.392884 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:16.393202 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:16.393491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] PATH_INFO: `/v3/users/9631ec5df93c42df9d2f60bedfef6507` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:16.394763 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9631ec5df93c42df9d2f60bedfef6507)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:16.396764 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:16.398497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:16.621598 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9631ec5df93c42df9d2f60bedfef6507'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:16.622250 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9631ec5df93c42df9d2f60bedfef6507'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:16.622980 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9631ec5df93c42df9d2f60bedfef6507'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:16.679814 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9631ec5df93c42df9d2f60bedfef6507'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:16.694231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9631ec5df93c42df9d2f60bedfef6507'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:16.707677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1289/2589] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:19:16 2022] DELETE /identity/v3/users/9631ec5df93c42df9d2f60bedfef6507 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:16.974836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6ea12-77a4-4864-9737-7bd1f327c714 service placement] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:16.987481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6ea12-77a4-4864-9737-7bd1f327c714 service placement] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:17.046860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:17.047736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:17.048023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:17.048300 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] PATH_INFO: `/v3/projects/019f72079acf4a74bd1a0706041e6678` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:17.050153 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=019f72079acf4a74bd1a0706041e6678)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:17.052083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:17.180820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '019f72079acf4a74bd1a0706041e6678', 'request_id': 'req-bb5f8556-2262-46a3-b906-62842e012de7'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:17.182015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:17.187664 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '019f72079acf4a74bd1a0706041e6678', 'request_id': 'req-bb5f8556-2262-46a3-b906-62842e012de7'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:17.194452 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '019f72079acf4a74bd1a0706041e6678', 'request_id': 'req-bb5f8556-2262-46a3-b906-62842e012de7'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:17.314337 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] The token cache is being invalidate because project 019f72079acf4a74bd1a0706041e6678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:19:17.316277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:17.316277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:17.317217 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1301/2590] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:19:16 2022] DELETE /identity/v3/projects/019f72079acf4a74bd1a0706041e6678 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:17.321228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:17.322673 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:17.337217 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b9d93-c3a6-4ca0-b44d-48e186c7711d admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:17.400228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:17.401118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:17.401406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:17.401682 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] PATH_INFO: `/v3/users/f0cf95af1cbb48f49ee2a20d64f693b4` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:17.402937 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0cf95af1cbb48f49ee2a20d64f693b4)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:17.404881 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:17.596080 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0cf95af1cbb48f49ee2a20d64f693b4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:17.596684 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0cf95af1cbb48f49ee2a20d64f693b4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:17.597287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0cf95af1cbb48f49ee2a20d64f693b4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:17.651266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0cf95af1cbb48f49ee2a20d64f693b4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:17.664444 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0cf95af1cbb48f49ee2a20d64f693b4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:17.672214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:17.677505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1290/2591] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:19:17 2022] DELETE /identity/v3/users/f0cf95af1cbb48f49ee2a20d64f693b4 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:17.949131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:17.954586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:17.968646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f8556-2262-46a3-b906-62842e012de7 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:18.030792 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:18.031672 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:18.031960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:18.032239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] PATH_INFO: `/v3/projects/98b61a648ed24c8b8caeb4e31475395d` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:18.033695 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b61a648ed24c8b8caeb4e31475395d)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:18.035601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:18.166556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b61a648ed24c8b8caeb4e31475395d', 'request_id': 'req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:18.173557 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b61a648ed24c8b8caeb4e31475395d', 'request_id': 'req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:18.184347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b61a648ed24c8b8caeb4e31475395d', 'request_id': 'req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:18.275888 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] The token cache is being invalidate because project 98b61a648ed24c8b8caeb4e31475395d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:19:18.281737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:18.281737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:18.281737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1302/2592] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:19:17 2022] DELETE /identity/v3/projects/98b61a648ed24c8b8caeb4e31475395d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:27.748063 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:28.271862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:38.731938 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:38.733006 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:38.792088 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ddd4b-f312-42f6-85bb-51735525eef0 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:38.843267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48051667-ce85-4b8a-b1c9-63b0f58738c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:38.844023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48051667-ce85-4b8a-b1c9-63b0f58738c8 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:38.844308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48051667-ce85-4b8a-b1c9-63b0f58738c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:38.844587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48051667-ce85-4b8a-b1c9-63b0f58738c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:38.846654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-48051667-ce85-4b8a-b1c9-63b0f58738c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:38.910845 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48051667-ce85-4b8a-b1c9-63b0f58738c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:38.912296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48051667-ce85-4b8a-b1c9-63b0f58738c8 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:38.922059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1291/2593] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:19:42.692145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:42.693612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:42.707674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5abfc-dcea-47d3-98cf-7c35a98e57e6 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:42.712428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d3472-d19d-4654-bd40-7878ba5db89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:42.713242 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d3472-d19d-4654-bd40-7878ba5db89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:42.713523 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d3472-d19d-4654-bd40-7878ba5db89c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:42.713800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d3472-d19d-4654-bd40-7878ba5db89c admin admin] PATH_INFO: `/v3/users/c99e02a845184f0aa206ba9f209ae4ef` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:42.715014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d3472-d19d-4654-bd40-7878ba5db89c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c99e02a845184f0aa206ba9f209ae4ef)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:42.716945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d3472-d19d-4654-bd40-7878ba5db89c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:42.904899 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-653d3472-d19d-4654-bd40-7878ba5db89c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c99e02a845184f0aa206ba9f209ae4ef'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:42.905572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-653d3472-d19d-4654-bd40-7878ba5db89c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c99e02a845184f0aa206ba9f209ae4ef'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:42.912875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-653d3472-d19d-4654-bd40-7878ba5db89c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c99e02a845184f0aa206ba9f209ae4ef'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:42.925901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-653d3472-d19d-4654-bd40-7878ba5db89c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c99e02a845184f0aa206ba9f209ae4ef'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:42.992434 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-653d3472-d19d-4654-bd40-7878ba5db89c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c99e02a845184f0aa206ba9f209ae4ef'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:42.998925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1303/2594] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:19:42 2022] DELETE /identity/v3/users/c99e02a845184f0aa206ba9f209ae4ef => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:43.279894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48051667-ce85-4b8a-b1c9-63b0f58738c8 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:43.293331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48051667-ce85-4b8a-b1c9-63b0f58738c8 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:43.355557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:43.356455 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:43.356742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:43.357020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570 admin admin] PATH_INFO: `/v3/projects/c2de35c4252b4a6fa55eb7e5d59e06cf` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:43.358198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2de35c4252b4a6fa55eb7e5d59e06cf)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:43.360173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:43.369137 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=100 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:43.549992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2de35c4252b4a6fa55eb7e5d59e06cf', 'request_id': 'req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:43.712486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2de35c4252b4a6fa55eb7e5d59e06cf', 'request_id': 'req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:43.718701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2de35c4252b4a6fa55eb7e5d59e06cf', 'request_id': 'req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:43.731595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570 admin admin] The token cache is being invalidate because project c2de35c4252b4a6fa55eb7e5d59e06cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:19:43.734222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:43.734222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-e7f1b062-1c82-48d9-a0c0-c9ddd2eb1570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:43.734222 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1292/2595] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:19:43 2022] DELETE /identity/v3/projects/c2de35c4252b4a6fa55eb7e5d59e06cf => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:43.740328 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e897b2-77bb-4368-972a-fa80bbd0badb None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:43.741106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e897b2-77bb-4368-972a-fa80bbd0badb None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:43.741218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e897b2-77bb-4368-972a-fa80bbd0badb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:43.760862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-83e897b2-77bb-4368-972a-fa80bbd0badb None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:43.762073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-83e897b2-77bb-4368-972a-fa80bbd0badb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:43.823376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1304/2596] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:43.831340 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235eb60-b538-4dfe-8d3b-49f5396ff124 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:43.831635 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235eb60-b538-4dfe-8d3b-49f5396ff124 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:43.831943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235eb60-b538-4dfe-8d3b-49f5396ff124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:43.852874 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-f235eb60-b538-4dfe-8d3b-49f5396ff124 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:43.854104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f235eb60-b538-4dfe-8d3b-49f5396ff124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:43.863786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1293/2597] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:43.867753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e897b2-77bb-4368-972a-fa80bbd0badb None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:43.877724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e897b2-77bb-4368-972a-fa80bbd0badb None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:43.882495 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7451a8-37bd-4be1-963e-68a16b5eb357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:43.883254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7451a8-37bd-4be1-963e-68a16b5eb357 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:43.883542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7451a8-37bd-4be1-963e-68a16b5eb357 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:43.883814 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7451a8-37bd-4be1-963e-68a16b5eb357 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:43.884530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7451a8-37bd-4be1-963e-68a16b5eb357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:19:43.884836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7451a8-37bd-4be1-963e-68a16b5eb357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:43.886452 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7451a8-37bd-4be1-963e-68a16b5eb357 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:43.898083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1305/2598] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:19:43.902599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235eb60-b538-4dfe-8d3b-49f5396ff124 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:43.912737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235eb60-b538-4dfe-8d3b-49f5396ff124 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:43.917515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89449060-74f1-4fe5-af39-209219cee9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:43.918173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89449060-74f1-4fe5-af39-209219cee9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:43.918493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89449060-74f1-4fe5-af39-209219cee9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:43.918784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89449060-74f1-4fe5-af39-209219cee9a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:43.919478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89449060-74f1-4fe5-af39-209219cee9a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:43.921376 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89449060-74f1-4fe5-af39-209219cee9a4 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:44.012819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1294/2599] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:19:43 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:19:44.017249 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7451a8-37bd-4be1-963e-68a16b5eb357 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:44.027829 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7451a8-37bd-4be1-963e-68a16b5eb357 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:44.087397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cd35d-eb2e-49c0-b69a-e9ddc7327dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:44.087992 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cd35d-eb2e-49c0-b69a-e9ddc7327dbe admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:44.088279 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cd35d-eb2e-49c0-b69a-e9ddc7327dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:44.088560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cd35d-eb2e-49c0-b69a-e9ddc7327dbe admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:44.089267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cd35d-eb2e-49c0-b69a-e9ddc7327dbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:44.091227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cd35d-eb2e-49c0-b69a-e9ddc7327dbe admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:44.099902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:44.206556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1306/2600] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:19:44 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:19:44.211133 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89449060-74f1-4fe5-af39-209219cee9a4 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:44.230899 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89449060-74f1-4fe5-af39-209219cee9a4 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:44.235857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a6e83-279b-4af9-a79d-a4c054e9d30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:44.236958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a6e83-279b-4af9-a79d-a4c054e9d30e admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:44.237251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a6e83-279b-4af9-a79d-a4c054e9d30e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:44.237642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a6e83-279b-4af9-a79d-a4c054e9d30e admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:44.238416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a6e83-279b-4af9-a79d-a4c054e9d30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:44.240034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a6e83-279b-4af9-a79d-a4c054e9d30e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:44.250750 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1295/2601] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:44.254956 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cd35d-eb2e-49c0-b69a-e9ddc7327dbe admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:44.265326 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cd35d-eb2e-49c0-b69a-e9ddc7327dbe admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:44.271365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba618c1-5b7f-45bb-844f-552e302248f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:44.272075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba618c1-5b7f-45bb-844f-552e302248f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:44.272497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba618c1-5b7f-45bb-844f-552e302248f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:44.272805 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba618c1-5b7f-45bb-844f-552e302248f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:44.273501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba618c1-5b7f-45bb-844f-552e302248f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:44.275168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba618c1-5b7f-45bb-844f-552e302248f4 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:44.287658 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1307/2602] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:44.292037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a6e83-279b-4af9-a79d-a4c054e9d30e admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:44.302908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a6e83-279b-4af9-a79d-a4c054e9d30e admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:44.308007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbedd655-1396-4b48-95c3-cb131626653b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:44.308901 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedd655-1396-4b48-95c3-cb131626653b admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:44.309192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedd655-1396-4b48-95c3-cb131626653b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:44.309475 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedd655-1396-4b48-95c3-cb131626653b admin admin] PATH_INFO: `/v3/projects/6df3ad763ab140198487b1a2aff4f84c/users/d48dc57c038c479a94167fddac7bd7c2/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:44.326118 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbedd655-1396-4b48-95c3-cb131626653b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df3ad763ab140198487b1a2aff4f84c, user_id=d48dc57c038c479a94167fddac7bd7c2, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:44.328835 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbedd655-1396-4b48-95c3-cb131626653b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:44.417528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1296/2603] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:19:44 2022] PUT /identity/v3/projects/6df3ad763ab140198487b1a2aff4f84c/users/d48dc57c038c479a94167fddac7bd7c2/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:44.430407 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0ef09-4d2d-4d0b-ae96-2ce806081744 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:44.430707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0ef09-4d2d-4d0b-ae96-2ce806081744 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:44.430993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0ef09-4d2d-4d0b-ae96-2ce806081744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:44.460277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-c3b0ef09-4d2d-4d0b-ae96-2ce806081744 None None] MFA Rules not processed for user `d48dc57c038c479a94167fddac7bd7c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:44.461506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c3b0ef09-4d2d-4d0b-ae96-2ce806081744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:44.539263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1308/2604] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:44.551419 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbedd655-1396-4b48-95c3-cb131626653b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:44.553242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-fbedd655-1396-4b48-95c3-cb131626653b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:44.631441 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbedd655-1396-4b48-95c3-cb131626653b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:44.637070 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78632068-2bab-4cdc-ba3f-d33886c90273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:44.637734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78632068-2bab-4cdc-ba3f-d33886c90273 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:44.638021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78632068-2bab-4cdc-ba3f-d33886c90273 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:44.638305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78632068-2bab-4cdc-ba3f-d33886c90273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:44.647682 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78632068-2bab-4cdc-ba3f-d33886c90273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:44.649286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78632068-2bab-4cdc-ba3f-d33886c90273 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:44.659290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1297/2605] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:19:47.822605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0ef09-4d2d-4d0b-ae96-2ce806081744 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:47.887422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0ef09-4d2d-4d0b-ae96-2ce806081744 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:47.892014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:47.893437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:47.893763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:47.894107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] PATH_INFO: `/v3/users/d48dc57c038c479a94167fddac7bd7c2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:47.895399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d48dc57c038c479a94167fddac7bd7c2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:47.897340 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:48.076574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd48dc57c038c479a94167fddac7bd7c2'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:48.077159 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd48dc57c038c479a94167fddac7bd7c2'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:48.085067 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd48dc57c038c479a94167fddac7bd7c2'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:48.095096 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:48.098219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd48dc57c038c479a94167fddac7bd7c2'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:48.188421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd48dc57c038c479a94167fddac7bd7c2'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:48.196093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1309/2606] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:19:47 2022] DELETE /identity/v3/users/d48dc57c038c479a94167fddac7bd7c2 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:48.200834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd0bb6-56b5-46ad-a488-89e373962c1c None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:48.201621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd0bb6-56b5-46ad-a488-89e373962c1c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:48.201904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd0bb6-56b5-46ad-a488-89e373962c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:48.222048 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e6bd0bb6-56b5-46ad-a488-89e373962c1c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:48.223298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e6bd0bb6-56b5-46ad-a488-89e373962c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:48.235683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:48.284567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1298/2607] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:48.294604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:48.296169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:48.355929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b7b54-0dbe-42d0-af88-8350f48e6fb0 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:48.359543 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754b728-2979-4513-bb43-aa17436a6089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:48.360210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754b728-2979-4513-bb43-aa17436a6089 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:48.360496 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754b728-2979-4513-bb43-aa17436a6089 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:48.360775 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754b728-2979-4513-bb43-aa17436a6089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:48.371800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754b728-2979-4513-bb43-aa17436a6089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:48.382783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754b728-2979-4513-bb43-aa17436a6089 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:48.383712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1310/2608] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:19:48.802451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd0bb6-56b5-46ad-a488-89e373962c1c None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:48.804001 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e6bd0bb6-56b5-46ad-a488-89e373962c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:48.817616 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd0bb6-56b5-46ad-a488-89e373962c1c None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:48.822771 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adbab9-03c5-4780-940d-32850b089f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:48.823665 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adbab9-03c5-4780-940d-32850b089f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:48.823957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adbab9-03c5-4780-940d-32850b089f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:48.824269 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adbab9-03c5-4780-940d-32850b089f14 admin admin] PATH_INFO: `/v3/projects/6df3ad763ab140198487b1a2aff4f84c` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:48.825485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adbab9-03c5-4780-940d-32850b089f14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6df3ad763ab140198487b1a2aff4f84c)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:48.827652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adbab9-03c5-4780-940d-32850b089f14 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:48.930376 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-16adbab9-03c5-4780-940d-32850b089f14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6df3ad763ab140198487b1a2aff4f84c', 'request_id': 'req-16adbab9-03c5-4780-940d-32850b089f14'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:48.993761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-16adbab9-03c5-4780-940d-32850b089f14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6df3ad763ab140198487b1a2aff4f84c', 'request_id': 'req-16adbab9-03c5-4780-940d-32850b089f14'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:49.000459 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-16adbab9-03c5-4780-940d-32850b089f14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6df3ad763ab140198487b1a2aff4f84c', 'request_id': 'req-16adbab9-03c5-4780-940d-32850b089f14'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:49.012981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-16adbab9-03c5-4780-940d-32850b089f14 admin admin] The token cache is being invalidate because project 6df3ad763ab140198487b1a2aff4f84c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:19:49.013277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-16adbab9-03c5-4780-940d-32850b089f14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:49.013978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-16adbab9-03c5-4780-940d-32850b089f14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:49.015560 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1299/2609] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:19:48 2022] DELETE /identity/v3/projects/6df3ad763ab140198487b1a2aff4f84c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:49.020507 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f4eea-1855-4cf4-9f3d-836f3f4ac09b None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:49.021020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f4eea-1855-4cf4-9f3d-836f3f4ac09b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:49.021129 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f4eea-1855-4cf4-9f3d-836f3f4ac09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:49.042194 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-913f4eea-1855-4cf4-9f3d-836f3f4ac09b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:49.043459 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-913f4eea-1855-4cf4-9f3d-836f3f4ac09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:49.103197 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1311/2610] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:49.111188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b08a1-e1a7-4136-a90f-6b9ce3cebdd7 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:49.116649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b08a1-e1a7-4136-a90f-6b9ce3cebdd7 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:49.116649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b08a1-e1a7-4136-a90f-6b9ce3cebdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:49.133050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-9a5b08a1-e1a7-4136-a90f-6b9ce3cebdd7 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:49.134288 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9a5b08a1-e1a7-4136-a90f-6b9ce3cebdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:49.142719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1300/2611] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:49.147152 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f4eea-1855-4cf4-9f3d-836f3f4ac09b None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:49.156994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f4eea-1855-4cf4-9f3d-836f3f4ac09b None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:49.162378 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b29c8d-417c-4322-8611-6586957a54e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:49.163579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b29c8d-417c-4322-8611-6586957a54e5 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:49.163579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b29c8d-417c-4322-8611-6586957a54e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:49.163579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b29c8d-417c-4322-8611-6586957a54e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:49.164307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b29c8d-417c-4322-8611-6586957a54e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:19:49.164718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b29c8d-417c-4322-8611-6586957a54e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:49.166548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b29c8d-417c-4322-8611-6586957a54e5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:49.179548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1312/2612] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:19:49.184160 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b08a1-e1a7-4136-a90f-6b9ce3cebdd7 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:49.194173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b08a1-e1a7-4136-a90f-6b9ce3cebdd7 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:49.198984 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19cdcc-ceaf-41ec-8e21-3da912be142e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:49.199647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19cdcc-ceaf-41ec-8e21-3da912be142e admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:49.199938 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19cdcc-ceaf-41ec-8e21-3da912be142e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:49.200223 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19cdcc-ceaf-41ec-8e21-3da912be142e admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:49.200928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19cdcc-ceaf-41ec-8e21-3da912be142e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:49.202926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19cdcc-ceaf-41ec-8e21-3da912be142e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:49.285826 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1301/2613] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:19:49 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:19:49.289974 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b29c8d-417c-4322-8611-6586957a54e5 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:49.301220 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b29c8d-417c-4322-8611-6586957a54e5 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:49.358948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6a020-8f1e-4e46-83a2-413137ab4edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:49.359626 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6a020-8f1e-4e46-83a2-413137ab4edf admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:49.359913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6a020-8f1e-4e46-83a2-413137ab4edf admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:49.360196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6a020-8f1e-4e46-83a2-413137ab4edf admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:49.360881 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6a020-8f1e-4e46-83a2-413137ab4edf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:49.363567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6a020-8f1e-4e46-83a2-413137ab4edf admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:49.483971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1313/2614] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:19:49 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:19:49.488658 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19cdcc-ceaf-41ec-8e21-3da912be142e admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:49.499344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19cdcc-ceaf-41ec-8e21-3da912be142e admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:49.504698 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e15de5-5453-4bb6-9d63-1e5f031efc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:49.505368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e15de5-5453-4bb6-9d63-1e5f031efc01 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:49.505705 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e15de5-5453-4bb6-9d63-1e5f031efc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:49.505985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e15de5-5453-4bb6-9d63-1e5f031efc01 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:49.506744 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e15de5-5453-4bb6-9d63-1e5f031efc01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:49.508357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e15de5-5453-4bb6-9d63-1e5f031efc01 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:49.519178 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1302/2615] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:49.523421 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6a020-8f1e-4e46-83a2-413137ab4edf admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:49.533979 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6a020-8f1e-4e46-83a2-413137ab4edf admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:49.538985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b0b9d-b79a-4e78-a55b-33e6f2fc21c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:49.539726 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b0b9d-b79a-4e78-a55b-33e6f2fc21c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:49.540102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b0b9d-b79a-4e78-a55b-33e6f2fc21c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:49.540390 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b0b9d-b79a-4e78-a55b-33e6f2fc21c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:49.541124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b0b9d-b79a-4e78-a55b-33e6f2fc21c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:49.542804 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b0b9d-b79a-4e78-a55b-33e6f2fc21c0 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:49.544167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:49.555324 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1314/2616] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:49.560037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e15de5-5453-4bb6-9d63-1e5f031efc01 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:49.571127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e15de5-5453-4bb6-9d63-1e5f031efc01 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:49.575800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c67da-f1eb-404d-90aa-314ae9cffb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:49.576682 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c67da-f1eb-404d-90aa-314ae9cffb95 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:49.576964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c67da-f1eb-404d-90aa-314ae9cffb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:49.577238 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c67da-f1eb-404d-90aa-314ae9cffb95 admin admin] PATH_INFO: `/v3/projects/57b17704d4384df894ad98072852dad9/users/c46f22aa8d004bd99e308aac55296f87/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:49.593422 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c67da-f1eb-404d-90aa-314ae9cffb95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57b17704d4384df894ad98072852dad9, user_id=c46f22aa8d004bd99e308aac55296f87, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:49.596120 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c67da-f1eb-404d-90aa-314ae9cffb95 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:49.675402 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1303/2617] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:19:49 2022] PUT /identity/v3/projects/57b17704d4384df894ad98072852dad9/users/c46f22aa8d004bd99e308aac55296f87/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:49.685989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c423183-d28f-4264-aac4-b69fb807d2b7 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:49.686279 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c423183-d28f-4264-aac4-b69fb807d2b7 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:49.693208 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c423183-d28f-4264-aac4-b69fb807d2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:49.715086 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-4c423183-d28f-4264-aac4-b69fb807d2b7 None None] MFA Rules not processed for user `c46f22aa8d004bd99e308aac55296f87`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:49.716302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-4c423183-d28f-4264-aac4-b69fb807d2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:49.780720 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1315/2618] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:49.785837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c67da-f1eb-404d-90aa-314ae9cffb95 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:49.830459 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:49.852942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c67da-f1eb-404d-90aa-314ae9cffb95 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:49.857653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92790e-9b83-4289-8ea2-668e1a2da711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:49.858339 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92790e-9b83-4289-8ea2-668e1a2da711 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:49.858641 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92790e-9b83-4289-8ea2-668e1a2da711 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:49.858920 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92790e-9b83-4289-8ea2-668e1a2da711 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:49.859648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92790e-9b83-4289-8ea2-668e1a2da711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:49.862447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92790e-9b83-4289-8ea2-668e1a2da711 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:49.990438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1304/2619] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:19:49 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:19:49.994954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c423183-d28f-4264-aac4-b69fb807d2b7 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:50.006548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c423183-d28f-4264-aac4-b69fb807d2b7 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:50.065622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d08f1-8d95-40b8-98a9-dc31eff482d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:50.066344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d08f1-8d95-40b8-98a9-dc31eff482d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:50.066667 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d08f1-8d95-40b8-98a9-dc31eff482d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:50.066977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d08f1-8d95-40b8-98a9-dc31eff482d4 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:50.067671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d08f1-8d95-40b8-98a9-dc31eff482d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:50.069586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d08f1-8d95-40b8-98a9-dc31eff482d4 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:50.206925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1316/2620] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:19:49 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:19:50.211052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92790e-9b83-4289-8ea2-668e1a2da711 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:50.222405 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92790e-9b83-4289-8ea2-668e1a2da711 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:50.227056 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6859c75-e7c6-45fa-9815-a51e6e95f658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:50.228219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6859c75-e7c6-45fa-9815-a51e6e95f658 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:50.228219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6859c75-e7c6-45fa-9815-a51e6e95f658 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:50.228429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6859c75-e7c6-45fa-9815-a51e6e95f658 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:50.229111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6859c75-e7c6-45fa-9815-a51e6e95f658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:50.230768 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6859c75-e7c6-45fa-9815-a51e6e95f658 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:50.241818 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1305/2621] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:50.246296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d08f1-8d95-40b8-98a9-dc31eff482d4 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:50.256915 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d08f1-8d95-40b8-98a9-dc31eff482d4 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:50.261602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cbf43-a4c1-407e-8f66-3c3803c2a938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:50.262531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cbf43-a4c1-407e-8f66-3c3803c2a938 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:50.262819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cbf43-a4c1-407e-8f66-3c3803c2a938 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:50.263101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cbf43-a4c1-407e-8f66-3c3803c2a938 admin admin] PATH_INFO: `/v3/projects/e420ae5d3c0149bfba809b5fe350e1ba/users/b9c2ac6d239e498dbb6e6e4125a37136/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:50.279428 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cbf43-a4c1-407e-8f66-3c3803c2a938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e420ae5d3c0149bfba809b5fe350e1ba, user_id=b9c2ac6d239e498dbb6e6e4125a37136, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:50.282070 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cbf43-a4c1-407e-8f66-3c3803c2a938 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:50.383422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1317/2622] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:19:50 2022] PUT /identity/v3/projects/e420ae5d3c0149bfba809b5fe350e1ba/users/b9c2ac6d239e498dbb6e6e4125a37136/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:50.387695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6859c75-e7c6-45fa-9815-a51e6e95f658 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:50.398860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6859c75-e7c6-45fa-9815-a51e6e95f658 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:50.458986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f111f-4ed3-4f62-9c26-ed68780382da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:50.459678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f111f-4ed3-4f62-9c26-ed68780382da admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:50.459969 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f111f-4ed3-4f62-9c26-ed68780382da admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:50.460252 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f111f-4ed3-4f62-9c26-ed68780382da admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:50.460986 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f111f-4ed3-4f62-9c26-ed68780382da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:50.462616 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f111f-4ed3-4f62-9c26-ed68780382da admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:50.472890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1306/2623] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:50.477184 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cbf43-a4c1-407e-8f66-3c3803c2a938 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:50.487720 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cbf43-a4c1-407e-8f66-3c3803c2a938 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:50.492690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ff149-4adf-44b0-bbbf-146b9c472eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:50.493572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff149-4adf-44b0-bbbf-146b9c472eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:50.493858 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff149-4adf-44b0-bbbf-146b9c472eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:50.494136 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff149-4adf-44b0-bbbf-146b9c472eb3 admin admin] PATH_INFO: `/v3/projects/e420ae5d3c0149bfba809b5fe350e1ba/users/b9c2ac6d239e498dbb6e6e4125a37136/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:50.496182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ff149-4adf-44b0-bbbf-146b9c472eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e420ae5d3c0149bfba809b5fe350e1ba, user_id=b9c2ac6d239e498dbb6e6e4125a37136, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:50.498852 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ff149-4adf-44b0-bbbf-146b9c472eb3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:50.509832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1318/2624] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:19:50 2022] PUT /identity/v3/projects/e420ae5d3c0149bfba809b5fe350e1ba/users/b9c2ac6d239e498dbb6e6e4125a37136/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:50.522580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613d7ce-1395-4d4f-99e3-5f55a409ff52 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:50.522877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613d7ce-1395-4d4f-99e3-5f55a409ff52 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:50.523162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613d7ce-1395-4d4f-99e3-5f55a409ff52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:50.553076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c613d7ce-1395-4d4f-99e3-5f55a409ff52 None None] MFA Rules not processed for user `b9c2ac6d239e498dbb6e6e4125a37136`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:50.554309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c613d7ce-1395-4d4f-99e3-5f55a409ff52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:50.623906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1307/2625] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:50.637022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ff149-4adf-44b0-bbbf-146b9c472eb3 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:50.638671 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7a5ff149-4adf-44b0-bbbf-146b9c472eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:50.684530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:50.699553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ff149-4adf-44b0-bbbf-146b9c472eb3 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:50.703460 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070dc6e-7258-4624-b82a-cd08b1174133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:50.707178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070dc6e-7258-4624-b82a-cd08b1174133 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:50.707178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070dc6e-7258-4624-b82a-cd08b1174133 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:50.707178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070dc6e-7258-4624-b82a-cd08b1174133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:50.714268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070dc6e-7258-4624-b82a-cd08b1174133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:50.715742 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070dc6e-7258-4624-b82a-cd08b1174133 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:50.725926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1319/2626] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:19:51.719709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c613d7ce-1395-4d4f-99e3-5f55a409ff52 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:51.759526 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:51.796207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c613d7ce-1395-4d4f-99e3-5f55a409ff52 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:51.801324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:51.805055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:51.805055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:51.805055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] PATH_INFO: `/v3/users/c46f22aa8d004bd99e308aac55296f87` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:51.805055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c46f22aa8d004bd99e308aac55296f87)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:51.806551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:52.015055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c46f22aa8d004bd99e308aac55296f87'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:52.015693 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c46f22aa8d004bd99e308aac55296f87'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:52.016581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c46f22aa8d004bd99e308aac55296f87'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:52.078030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c46f22aa8d004bd99e308aac55296f87'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:52.094717 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c46f22aa8d004bd99e308aac55296f87'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:52.110203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1308/2627] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:19:51 2022] DELETE /identity/v3/users/c46f22aa8d004bd99e308aac55296f87 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:52.115411 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951b9a5-8a33-459d-98ee-c19a9fd562f1 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:52.116283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951b9a5-8a33-459d-98ee-c19a9fd562f1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:52.116283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951b9a5-8a33-459d-98ee-c19a9fd562f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:52.138170 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-a951b9a5-8a33-459d-98ee-c19a9fd562f1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:52.139511 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a951b9a5-8a33-459d-98ee-c19a9fd562f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:52.213901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1320/2628] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:52.225865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:52.227513 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:52.293403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56569819-97a3-47a7-b8f5-fd65a4e00131 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:52.295012 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3423c7-0aac-4e73-85ee-e3f542946d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:52.295670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3423c7-0aac-4e73-85ee-e3f542946d20 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:52.295961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3423c7-0aac-4e73-85ee-e3f542946d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:52.296241 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3423c7-0aac-4e73-85ee-e3f542946d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:52.305748 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3423c7-0aac-4e73-85ee-e3f542946d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:52.307253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3423c7-0aac-4e73-85ee-e3f542946d20 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:52.318196 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1309/2629] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:19:52.822802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951b9a5-8a33-459d-98ee-c19a9fd562f1 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:52.824255 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a951b9a5-8a33-459d-98ee-c19a9fd562f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:52.837004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951b9a5-8a33-459d-98ee-c19a9fd562f1 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:52.842534 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:52.843442 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:52.843728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:52.844006 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] PATH_INFO: `/v3/projects/57b17704d4384df894ad98072852dad9` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:52.845203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57b17704d4384df894ad98072852dad9)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:52.847244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:52.948698 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57b17704d4384df894ad98072852dad9', 'request_id': 'req-57c8e2df-5356-4020-9d46-6f9e22959940'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:52.955981 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57b17704d4384df894ad98072852dad9', 'request_id': 'req-57c8e2df-5356-4020-9d46-6f9e22959940'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:52.961757 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57b17704d4384df894ad98072852dad9', 'request_id': 'req-57c8e2df-5356-4020-9d46-6f9e22959940'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:53.045259 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] The token cache is being invalidate because project 57b17704d4384df894ad98072852dad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:19:53.045552 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:53.046198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:53.047880 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1321/2630] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:19:52 2022] DELETE /identity/v3/projects/57b17704d4384df894ad98072852dad9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:53.052126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3423c7-0aac-4e73-85ee-e3f542946d20 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:53.062033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3423c7-0aac-4e73-85ee-e3f542946d20 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:53.118038 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:53.124917 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d1126-fd2c-4ae4-a4fb-daf594c075ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:53.125800 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d1126-fd2c-4ae4-a4fb-daf594c075ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:53.126083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d1126-fd2c-4ae4-a4fb-daf594c075ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:53.126383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d1126-fd2c-4ae4-a4fb-daf594c075ac admin admin] PATH_INFO: `/v3/users/b9c2ac6d239e498dbb6e6e4125a37136` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:53.128353 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d1126-fd2c-4ae4-a4fb-daf594c075ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c2ac6d239e498dbb6e6e4125a37136)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:53.130266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d1126-fd2c-4ae4-a4fb-daf594c075ac admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:53.314895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c85d1126-fd2c-4ae4-a4fb-daf594c075ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c2ac6d239e498dbb6e6e4125a37136'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:53.315614 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c85d1126-fd2c-4ae4-a4fb-daf594c075ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c2ac6d239e498dbb6e6e4125a37136'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:53.316271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c85d1126-fd2c-4ae4-a4fb-daf594c075ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c2ac6d239e498dbb6e6e4125a37136'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:53.398062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c85d1126-fd2c-4ae4-a4fb-daf594c075ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c2ac6d239e498dbb6e6e4125a37136'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:53.411756 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-c85d1126-fd2c-4ae4-a4fb-daf594c075ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c2ac6d239e498dbb6e6e4125a37136'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:53.424570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1310/2631] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:19:53 2022] DELETE /identity/v3/users/b9c2ac6d239e498dbb6e6e4125a37136 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:53.851578 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:53.853090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:53.868210 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8e2df-5356-4020-9d46-6f9e22959940 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:53.929735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17cdaf-f644-4ad6-b575-f904ad267c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:53.930647 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17cdaf-f644-4ad6-b575-f904ad267c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:53.930937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17cdaf-f644-4ad6-b575-f904ad267c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:53.931265 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17cdaf-f644-4ad6-b575-f904ad267c6f admin admin] PATH_INFO: `/v3/projects/e420ae5d3c0149bfba809b5fe350e1ba` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:53.932590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17cdaf-f644-4ad6-b575-f904ad267c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e420ae5d3c0149bfba809b5fe350e1ba)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:53.934512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17cdaf-f644-4ad6-b575-f904ad267c6f admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:53.941698 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:54.045663 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ec17cdaf-f644-4ad6-b575-f904ad267c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e420ae5d3c0149bfba809b5fe350e1ba', 'request_id': 'req-ec17cdaf-f644-4ad6-b575-f904ad267c6f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:54.052665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ec17cdaf-f644-4ad6-b575-f904ad267c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e420ae5d3c0149bfba809b5fe350e1ba', 'request_id': 'req-ec17cdaf-f644-4ad6-b575-f904ad267c6f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:54.058521 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ec17cdaf-f644-4ad6-b575-f904ad267c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e420ae5d3c0149bfba809b5fe350e1ba', 'request_id': 'req-ec17cdaf-f644-4ad6-b575-f904ad267c6f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:54.141905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ec17cdaf-f644-4ad6-b575-f904ad267c6f admin admin] The token cache is being invalidate because project e420ae5d3c0149bfba809b5fe350e1ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:19:54.142213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ec17cdaf-f644-4ad6-b575-f904ad267c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:54.142940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-ec17cdaf-f644-4ad6-b575-f904ad267c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:54.144604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1322/2632] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:19:53 2022] DELETE /identity/v3/projects/e420ae5d3c0149bfba809b5fe350e1ba => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:54.149999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae6803-0030-409a-98da-6f0138ca687e None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:54.152519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae6803-0030-409a-98da-6f0138ca687e None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:54.152519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae6803-0030-409a-98da-6f0138ca687e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:54.172041 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b2ae6803-0030-409a-98da-6f0138ca687e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:54.173258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b2ae6803-0030-409a-98da-6f0138ca687e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:54.235595 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1311/2633] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:19:54.244116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265d096-2a9d-4d71-ab2b-627acf46557d None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:54.244416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265d096-2a9d-4d71-ab2b-627acf46557d None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:54.244706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265d096-2a9d-4d71-ab2b-627acf46557d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:54.264465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-3265d096-2a9d-4d71-ab2b-627acf46557d None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:54.265690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3265d096-2a9d-4d71-ab2b-627acf46557d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:54.274037 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1323/2634] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:54.278541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae6803-0030-409a-98da-6f0138ca687e None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:54.289141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae6803-0030-409a-98da-6f0138ca687e None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:54.293719 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777017d-fc10-47cd-893d-cf38777e967d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:54.295469 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777017d-fc10-47cd-893d-cf38777e967d admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:54.295469 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777017d-fc10-47cd-893d-cf38777e967d admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:54.295469 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777017d-fc10-47cd-893d-cf38777e967d admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:54.296011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777017d-fc10-47cd-893d-cf38777e967d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:19:54.296108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777017d-fc10-47cd-893d-cf38777e967d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:54.297707 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777017d-fc10-47cd-893d-cf38777e967d admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:54.310677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1312/2635] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:19:54.315303 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265d096-2a9d-4d71-ab2b-627acf46557d None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:54.324594 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265d096-2a9d-4d71-ab2b-627acf46557d None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:54.329567 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a3007-3333-47fe-a9d3-cd14a9ef4664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:54.330271 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a3007-3333-47fe-a9d3-cd14a9ef4664 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:54.330624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a3007-3333-47fe-a9d3-cd14a9ef4664 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:54.330911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a3007-3333-47fe-a9d3-cd14a9ef4664 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:54.331617 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a3007-3333-47fe-a9d3-cd14a9ef4664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:54.333528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a3007-3333-47fe-a9d3-cd14a9ef4664 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:54.442042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1324/2636] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:19:54 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:19:54.446311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777017d-fc10-47cd-893d-cf38777e967d admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:54.456836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777017d-fc10-47cd-893d-cf38777e967d admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:54.513739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ddd43-32ba-4147-805d-d313f31aeccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:54.514364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ddd43-32ba-4147-805d-d313f31aeccd admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:54.514667 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ddd43-32ba-4147-805d-d313f31aeccd admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:54.514928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ddd43-32ba-4147-805d-d313f31aeccd admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:54.515623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ddd43-32ba-4147-805d-d313f31aeccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:54.517582 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ddd43-32ba-4147-805d-d313f31aeccd admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:54.526925 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:54.646252 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1313/2637] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:19:54 2022] POST /identity/v3/users => generated 283 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:19:54.650615 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a3007-3333-47fe-a9d3-cd14a9ef4664 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:54.662435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a3007-3333-47fe-a9d3-cd14a9ef4664 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:54.667271 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbca7a-e895-4e76-9221-c2894d27170c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:54.667777 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbca7a-e895-4e76-9221-c2894d27170c admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:54.668068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbca7a-e895-4e76-9221-c2894d27170c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:54.668347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbca7a-e895-4e76-9221-c2894d27170c admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:54.669091 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbca7a-e895-4e76-9221-c2894d27170c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:54.670794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbca7a-e895-4e76-9221-c2894d27170c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:54.681938 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1325/2638] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:54.686568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ddd43-32ba-4147-805d-d313f31aeccd admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:54.697152 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ddd43-32ba-4147-805d-d313f31aeccd admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:54.701915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a255a1-3d20-4faf-b779-eda7e68a13e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:54.702621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a255a1-3d20-4faf-b779-eda7e68a13e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:54.702936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a255a1-3d20-4faf-b779-eda7e68a13e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:54.703215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a255a1-3d20-4faf-b779-eda7e68a13e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:54.704021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a255a1-3d20-4faf-b779-eda7e68a13e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:54.706174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a255a1-3d20-4faf-b779-eda7e68a13e2 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:54.717134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1314/2639] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:54.722088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbca7a-e895-4e76-9221-c2894d27170c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:54.732244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbca7a-e895-4e76-9221-c2894d27170c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:54.737059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d2e6e-c970-414b-a3e2-5edf4029526a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:54.737942 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d2e6e-c970-414b-a3e2-5edf4029526a admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:54.738229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d2e6e-c970-414b-a3e2-5edf4029526a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:54.738539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d2e6e-c970-414b-a3e2-5edf4029526a admin admin] PATH_INFO: `/v3/projects/529126e172954106b86d4c5f201b7737/users/afc997f4c19f4830a329410fb55fc136/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:54.754158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d2e6e-c970-414b-a3e2-5edf4029526a admin admin] RBAC: Authorizing `identity:create_grant(project_id=529126e172954106b86d4c5f201b7737, user_id=afc997f4c19f4830a329410fb55fc136, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:54.756845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d2e6e-c970-414b-a3e2-5edf4029526a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:54.828653 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1326/2640] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:19:54 2022] PUT /identity/v3/projects/529126e172954106b86d4c5f201b7737/users/afc997f4c19f4830a329410fb55fc136/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:54.841162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb6470-1f14-4592-ba5d-2e35d6256efc None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:54.841459 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb6470-1f14-4592-ba5d-2e35d6256efc None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:54.841772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb6470-1f14-4592-ba5d-2e35d6256efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:54.871999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-29eb6470-1f14-4592-ba5d-2e35d6256efc None None] MFA Rules not processed for user `afc997f4c19f4830a329410fb55fc136`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:54.873278 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-29eb6470-1f14-4592-ba5d-2e35d6256efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:54.937566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1315/2641] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:54.943008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d2e6e-c970-414b-a3e2-5edf4029526a admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:55.006263 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d2e6e-c970-414b-a3e2-5edf4029526a admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:55.010944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181a3b9-5666-4a09-84b0-7420449ad2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:55.011607 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181a3b9-5666-4a09-84b0-7420449ad2cc admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:55.011894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181a3b9-5666-4a09-84b0-7420449ad2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:55.012172 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181a3b9-5666-4a09-84b0-7420449ad2cc admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:55.012858 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181a3b9-5666-4a09-84b0-7420449ad2cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:55.014776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181a3b9-5666-4a09-84b0-7420449ad2cc admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:55.108124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1327/2642] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:19:54 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:19:55.112955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb6470-1f14-4592-ba5d-2e35d6256efc None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:55.124364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb6470-1f14-4592-ba5d-2e35d6256efc None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:55.184414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e4032-cd9f-413b-a962-144a84643ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:55.185394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e4032-cd9f-413b-a962-144a84643ded admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:55.185394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e4032-cd9f-413b-a962-144a84643ded admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:55.185681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e4032-cd9f-413b-a962-144a84643ded admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:55.186396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e4032-cd9f-413b-a962-144a84643ded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:55.188303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e4032-cd9f-413b-a962-144a84643ded admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:55.291662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1316/2643] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:19:55 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:19:55.295883 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181a3b9-5666-4a09-84b0-7420449ad2cc admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:55.307405 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181a3b9-5666-4a09-84b0-7420449ad2cc admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:55.312042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a3cfb-a6c3-42fd-aa1c-5a698f9801f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:55.312714 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a3cfb-a6c3-42fd-aa1c-5a698f9801f7 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:55.313004 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a3cfb-a6c3-42fd-aa1c-5a698f9801f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:55.313286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a3cfb-a6c3-42fd-aa1c-5a698f9801f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:55.314015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a3cfb-a6c3-42fd-aa1c-5a698f9801f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:55.315691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a3cfb-a6c3-42fd-aa1c-5a698f9801f7 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:55.326817 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1328/2644] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:55.332248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e4032-cd9f-413b-a962-144a84643ded admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:55.342892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e4032-cd9f-413b-a962-144a84643ded admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:55.348443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde0fb3-4430-41ae-9ce6-52d84d6efef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:55.349331 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde0fb3-4430-41ae-9ce6-52d84d6efef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:55.349622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde0fb3-4430-41ae-9ce6-52d84d6efef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:55.349905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde0fb3-4430-41ae-9ce6-52d84d6efef9 admin admin] PATH_INFO: `/v3/projects/f21a922c4508475aa643325c671f03bc/users/d66030af80ad49e38acaca7d01ca5df8/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:55.366315 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde0fb3-4430-41ae-9ce6-52d84d6efef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f21a922c4508475aa643325c671f03bc, user_id=d66030af80ad49e38acaca7d01ca5df8, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:55.368977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde0fb3-4430-41ae-9ce6-52d84d6efef9 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:55.435587 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1317/2645] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:19:55 2022] PUT /identity/v3/projects/f21a922c4508475aa643325c671f03bc/users/d66030af80ad49e38acaca7d01ca5df8/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:55.440001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a3cfb-a6c3-42fd-aa1c-5a698f9801f7 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:55.446420 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:55.450363 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a3cfb-a6c3-42fd-aa1c-5a698f9801f7 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:55.507499 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c01a-ff2e-4913-82fa-f6677d992789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:55.508221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c01a-ff2e-4913-82fa-f6677d992789 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:55.508546 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c01a-ff2e-4913-82fa-f6677d992789 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:55.508828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c01a-ff2e-4913-82fa-f6677d992789 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:55.509684 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c01a-ff2e-4913-82fa-f6677d992789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:55.511331 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c01a-ff2e-4913-82fa-f6677d992789 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:55.521843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1329/2646] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:55.526085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde0fb3-4430-41ae-9ce6-52d84d6efef9 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:55.538428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde0fb3-4430-41ae-9ce6-52d84d6efef9 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:55.543197 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6f2f2-3a93-4a48-8b76-c66c2a375314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:55.547520 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6f2f2-3a93-4a48-8b76-c66c2a375314 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:55.547520 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6f2f2-3a93-4a48-8b76-c66c2a375314 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:55.547520 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6f2f2-3a93-4a48-8b76-c66c2a375314 admin admin] PATH_INFO: `/v3/projects/f21a922c4508475aa643325c671f03bc/users/d66030af80ad49e38acaca7d01ca5df8/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:55.547520 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6f2f2-3a93-4a48-8b76-c66c2a375314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f21a922c4508475aa643325c671f03bc, user_id=d66030af80ad49e38acaca7d01ca5df8, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:55.549701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6f2f2-3a93-4a48-8b76-c66c2a375314 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:55.560356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1318/2647] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:19:55 2022] PUT /identity/v3/projects/f21a922c4508475aa643325c671f03bc/users/d66030af80ad49e38acaca7d01ca5df8/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:55.570792 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f3127-0d99-449d-98c0-2404a04e9a53 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:55.571192 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f3127-0d99-449d-98c0-2404a04e9a53 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:55.571405 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f3127-0d99-449d-98c0-2404a04e9a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:55.605292 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-4f9f3127-0d99-449d-98c0-2404a04e9a53 None None] MFA Rules not processed for user `d66030af80ad49e38acaca7d01ca5df8`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:55.606630 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-4f9f3127-0d99-449d-98c0-2404a04e9a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:55.676046 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1330/2648] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:55 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:55.687309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6f2f2-3a93-4a48-8b76-c66c2a375314 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:55.689084 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d1c6f2f2-3a93-4a48-8b76-c66c2a375314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:55.722978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:55.755821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6f2f2-3a93-4a48-8b76-c66c2a375314 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:55.759538 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11666d-df13-44c4-9b18-b1b4ce84b6b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:55.760276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11666d-df13-44c4-9b18-b1b4ce84b6b5 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:55.760572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11666d-df13-44c4-9b18-b1b4ce84b6b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:55.760869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11666d-df13-44c4-9b18-b1b4ce84b6b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:55.771090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11666d-df13-44c4-9b18-b1b4ce84b6b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:55.772525 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11666d-df13-44c4-9b18-b1b4ce84b6b5 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:55.784098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1319/2649] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:55 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:19:56.656360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f3127-0d99-449d-98c0-2404a04e9a53 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:56.721163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f3127-0d99-449d-98c0-2404a04e9a53 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:56.727040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:56.728215 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:56.728506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:56.730691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] PATH_INFO: `/v3/users/afc997f4c19f4830a329410fb55fc136` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:56.732002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afc997f4c19f4830a329410fb55fc136)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:56.733977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:56.834447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:56.943214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afc997f4c19f4830a329410fb55fc136'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:56.943897 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afc997f4c19f4830a329410fb55fc136'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:56.951034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afc997f4c19f4830a329410fb55fc136'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:56.962951 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afc997f4c19f4830a329410fb55fc136'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:57.014656 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afc997f4c19f4830a329410fb55fc136'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:57.021066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1331/2650] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:19:56 2022] DELETE /identity/v3/users/afc997f4c19f4830a329410fb55fc136 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:57.025700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48ec09-80f6-4a2f-b6f6-5e4627802bfd None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:57.026234 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48ec09-80f6-4a2f-b6f6-5e4627802bfd None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:57.026363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48ec09-80f6-4a2f-b6f6-5e4627802bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:57.046082 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-bf48ec09-80f6-4a2f-b6f6-5e4627802bfd None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:57.047341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bf48ec09-80f6-4a2f-b6f6-5e4627802bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:57.113389 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1320/2651] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:57.124248 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:57.126229 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:57.191364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595115e-534a-4942-b070-a07171f59104 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:57.195093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f346e-7b17-4f1b-bfa8-94d2597a9d13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:57.195680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f346e-7b17-4f1b-bfa8-94d2597a9d13 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:57.195961 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f346e-7b17-4f1b-bfa8-94d2597a9d13 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:57.196234 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f346e-7b17-4f1b-bfa8-94d2597a9d13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:57.205576 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f346e-7b17-4f1b-bfa8-94d2597a9d13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:57.207039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f346e-7b17-4f1b-bfa8-94d2597a9d13 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:57.216593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1332/2652] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:19:57.692799 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48ec09-80f6-4a2f-b6f6-5e4627802bfd None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:57.694371 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-bf48ec09-80f6-4a2f-b6f6-5e4627802bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:57.708090 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48ec09-80f6-4a2f-b6f6-5e4627802bfd None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:57.713365 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:57.713722 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:57.714016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:57.714299 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] PATH_INFO: `/v3/projects/529126e172954106b86d4c5f201b7737` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:57.715776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=529126e172954106b86d4c5f201b7737)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:57.717761 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:57.903979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '529126e172954106b86d4c5f201b7737', 'request_id': 'req-526be5d6-116b-490a-b9c1-5b7fffbefd40'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:58.005361 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '529126e172954106b86d4c5f201b7737', 'request_id': 'req-526be5d6-116b-490a-b9c1-5b7fffbefd40'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:58.011767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '529126e172954106b86d4c5f201b7737', 'request_id': 'req-526be5d6-116b-490a-b9c1-5b7fffbefd40'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:58.023264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] The token cache is being invalidate because project 529126e172954106b86d4c5f201b7737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:19:58.023562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:58.024256 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:58.025801 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1321/2653] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:19:57 2022] DELETE /identity/v3/projects/529126e172954106b86d4c5f201b7737 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:58.030002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f346e-7b17-4f1b-bfa8-94d2597a9d13 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:58.040657 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f346e-7b17-4f1b-bfa8-94d2597a9d13 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:58.097791 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7e84b-3907-4976-839e-d183683c1a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:58.098732 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7e84b-3907-4976-839e-d183683c1a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:58.099017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7e84b-3907-4976-839e-d183683c1a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:58.099293 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7e84b-3907-4976-839e-d183683c1a26 admin admin] PATH_INFO: `/v3/users/d66030af80ad49e38acaca7d01ca5df8` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:58.100488 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7e84b-3907-4976-839e-d183683c1a26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d66030af80ad49e38acaca7d01ca5df8)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:58.102498 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7e84b-3907-4976-839e-d183683c1a26 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:58.316390 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-aae7e84b-3907-4976-839e-d183683c1a26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd66030af80ad49e38acaca7d01ca5df8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:58.316980 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-aae7e84b-3907-4976-839e-d183683c1a26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd66030af80ad49e38acaca7d01ca5df8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:58.319175 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:58.324512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-aae7e84b-3907-4976-839e-d183683c1a26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd66030af80ad49e38acaca7d01ca5df8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:58.337861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-aae7e84b-3907-4976-839e-d183683c1a26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd66030af80ad49e38acaca7d01ca5df8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:58.480169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-aae7e84b-3907-4976-839e-d183683c1a26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd66030af80ad49e38acaca7d01ca5df8'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:58.487053 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1333/2654] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:19:58 2022] DELETE /identity/v3/users/d66030af80ad49e38acaca7d01ca5df8 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:58.931226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:58.932681 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:58.947559 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526be5d6-116b-490a-b9c1-5b7fffbefd40 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:58.995360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:19:59.007872 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db73c2c-25cf-4995-9d0b-a87a2b11e650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:59.008752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db73c2c-25cf-4995-9d0b-a87a2b11e650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:59.009039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db73c2c-25cf-4995-9d0b-a87a2b11e650 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:59.009320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db73c2c-25cf-4995-9d0b-a87a2b11e650 admin admin] PATH_INFO: `/v3/projects/f21a922c4508475aa643325c671f03bc` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:59.010613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db73c2c-25cf-4995-9d0b-a87a2b11e650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f21a922c4508475aa643325c671f03bc)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:59.012612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db73c2c-25cf-4995-9d0b-a87a2b11e650 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:59.200143 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-0db73c2c-25cf-4995-9d0b-a87a2b11e650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f21a922c4508475aa643325c671f03bc', 'request_id': 'req-0db73c2c-25cf-4995-9d0b-a87a2b11e650'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:59.284163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-0db73c2c-25cf-4995-9d0b-a87a2b11e650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f21a922c4508475aa643325c671f03bc', 'request_id': 'req-0db73c2c-25cf-4995-9d0b-a87a2b11e650'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:59.291803 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-0db73c2c-25cf-4995-9d0b-a87a2b11e650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f21a922c4508475aa643325c671f03bc', 'request_id': 'req-0db73c2c-25cf-4995-9d0b-a87a2b11e650'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:59.304234 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-0db73c2c-25cf-4995-9d0b-a87a2b11e650 admin admin] The token cache is being invalidate because project f21a922c4508475aa643325c671f03bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:19:59.304579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-0db73c2c-25cf-4995-9d0b-a87a2b11e650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:59.305300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-0db73c2c-25cf-4995-9d0b-a87a2b11e650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:19:59.307083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1322/2655] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:19:58 2022] DELETE /identity/v3/projects/f21a922c4508475aa643325c671f03bc => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:19:59.312472 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98029e0f-a803-4c13-bb5c-f50a40183d09 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:59.312988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98029e0f-a803-4c13-bb5c-f50a40183d09 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:59.313093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98029e0f-a803-4c13-bb5c-f50a40183d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:59.334075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-98029e0f-a803-4c13-bb5c-f50a40183d09 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:59.335325 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-98029e0f-a803-4c13-bb5c-f50a40183d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:59.396131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1334/2656] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:59.404131 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9deb8f-a875-4844-a711-bae09ecd2740 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:59.406664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9deb8f-a875-4844-a711-bae09ecd2740 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:59.406664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9deb8f-a875-4844-a711-bae09ecd2740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:59.426271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-2e9deb8f-a875-4844-a711-bae09ecd2740 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:19:59.427597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2e9deb8f-a875-4844-a711-bae09ecd2740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:19:59.435824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1323/2657] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:19:59.440660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98029e0f-a803-4c13-bb5c-f50a40183d09 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:59.450403 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98029e0f-a803-4c13-bb5c-f50a40183d09 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:59.455477 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06355736-20f7-4378-a27e-92d7e5adfbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:59.456133 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06355736-20f7-4378-a27e-92d7e5adfbb6 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:59.456419 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06355736-20f7-4378-a27e-92d7e5adfbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:59.456697 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06355736-20f7-4378-a27e-92d7e5adfbb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:59.457422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06355736-20f7-4378-a27e-92d7e5adfbb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:19:59.457734 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06355736-20f7-4378-a27e-92d7e5adfbb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:59.459365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06355736-20f7-4378-a27e-92d7e5adfbb6 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:59.473830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1335/2658] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:19:59.478347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9deb8f-a875-4844-a711-bae09ecd2740 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:59.488177 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9deb8f-a875-4844-a711-bae09ecd2740 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:59.493264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf612328-72b1-4136-b7f3-74755a78fcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:59.493933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf612328-72b1-4136-b7f3-74755a78fcbd admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:59.494224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf612328-72b1-4136-b7f3-74755a78fcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:59.494541 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf612328-72b1-4136-b7f3-74755a78fcbd admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:59.495295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf612328-72b1-4136-b7f3-74755a78fcbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:59.497209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf612328-72b1-4136-b7f3-74755a78fcbd admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:59.622355 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1324/2659] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:19:59 2022] POST /identity/v3/projects => generated 357 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:19:59.626728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06355736-20f7-4378-a27e-92d7e5adfbb6 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:59.637608 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06355736-20f7-4378-a27e-92d7e5adfbb6 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:59.694609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30dd51-0a81-455c-bd7d-a660a58ad271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:59.695278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30dd51-0a81-455c-bd7d-a660a58ad271 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:59.695563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30dd51-0a81-455c-bd7d-a660a58ad271 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:59.695841 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30dd51-0a81-455c-bd7d-a660a58ad271 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:59.696521 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30dd51-0a81-455c-bd7d-a660a58ad271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:59.698441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30dd51-0a81-455c-bd7d-a660a58ad271 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:59.787136 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1336/2660] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:19:59 2022] POST /identity/v3/users => generated 279 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:19:59.791218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf612328-72b1-4136-b7f3-74755a78fcbd admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:59.802175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf612328-72b1-4136-b7f3-74755a78fcbd admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:59.806797 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17b49-7fbd-4a3a-86c2-95bb7bc0bfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:59.807521 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17b49-7fbd-4a3a-86c2-95bb7bc0bfa3 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:59.807811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17b49-7fbd-4a3a-86c2-95bb7bc0bfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:59.808095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17b49-7fbd-4a3a-86c2-95bb7bc0bfa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:59.808832 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc17b49-7fbd-4a3a-86c2-95bb7bc0bfa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:59.810473 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc17b49-7fbd-4a3a-86c2-95bb7bc0bfa3 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:59.822235 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1325/2661] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:59.827926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30dd51-0a81-455c-bd7d-a660a58ad271 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:59.837572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30dd51-0a81-455c-bd7d-a660a58ad271 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:59.843833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1be86-0f37-4775-9e45-565fd2d4042e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:59.844518 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1be86-0f37-4775-9e45-565fd2d4042e admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:59.844814 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1be86-0f37-4775-9e45-565fd2d4042e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:59.845099 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1be86-0f37-4775-9e45-565fd2d4042e admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:59.845893 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1be86-0f37-4775-9e45-565fd2d4042e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:59.847727 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1be86-0f37-4775-9e45-565fd2d4042e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:59.858987 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1337/2662] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:19:59.863891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17b49-7fbd-4a3a-86c2-95bb7bc0bfa3 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:19:59.874791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17b49-7fbd-4a3a-86c2-95bb7bc0bfa3 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:19:59.880323 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caca81-35e4-41b4-9d2d-a89de63fcf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:19:59.881212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caca81-35e4-41b4-9d2d-a89de63fcf8b admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:59.881499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caca81-35e4-41b4-9d2d-a89de63fcf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:59.881779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caca81-35e4-41b4-9d2d-a89de63fcf8b admin admin] PATH_INFO: `/v3/projects/d99194853de64e88985402c874bf90dd/users/baead5cd60bf4471a55a705347b13460/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:19:59.896677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49caca81-35e4-41b4-9d2d-a89de63fcf8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d99194853de64e88985402c874bf90dd, user_id=baead5cd60bf4471a55a705347b13460, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:19:59.899333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49caca81-35e4-41b4-9d2d-a89de63fcf8b admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:19:59.979206 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1326/2663] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:19:59 2022] PUT /identity/v3/projects/d99194853de64e88985402c874bf90dd/users/baead5cd60bf4471a55a705347b13460/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:19:59.991446 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359a0f4-150b-4f6b-a956-89bac770f912 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:19:59.991735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359a0f4-150b-4f6b-a956-89bac770f912 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:19:59.992016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359a0f4-150b-4f6b-a956-89bac770f912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:00.021022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-5359a0f4-150b-4f6b-a956-89bac770f912 None None] MFA Rules not processed for user `baead5cd60bf4471a55a705347b13460`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:00.022286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5359a0f4-150b-4f6b-a956-89bac770f912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:00.042376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:00.087386 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1338/2664] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:19:59 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:00.092039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caca81-35e4-41b4-9d2d-a89de63fcf8b admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:00.164058 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caca81-35e4-41b4-9d2d-a89de63fcf8b admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:00.168889 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480bf5f-e7fb-4f0d-82ad-2afb1da76c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:00.177695 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480bf5f-e7fb-4f0d-82ad-2afb1da76c08 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:00.178016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480bf5f-e7fb-4f0d-82ad-2afb1da76c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:00.178266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480bf5f-e7fb-4f0d-82ad-2afb1da76c08 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:00.179020 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480bf5f-e7fb-4f0d-82ad-2afb1da76c08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:00.181110 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480bf5f-e7fb-4f0d-82ad-2afb1da76c08 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:00.246292 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:00.253935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1327/2665] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:20:00 2022] POST /identity/v3/projects => generated 357 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:20:00.258055 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359a0f4-150b-4f6b-a956-89bac770f912 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:00.270171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359a0f4-150b-4f6b-a956-89bac770f912 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:00.328124 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b930021-15bd-4d3f-9f55-b458fa9270e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:00.328770 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b930021-15bd-4d3f-9f55-b458fa9270e7 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:00.329040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b930021-15bd-4d3f-9f55-b458fa9270e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:00.329346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b930021-15bd-4d3f-9f55-b458fa9270e7 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:00.330047 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b930021-15bd-4d3f-9f55-b458fa9270e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:00.332582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b930021-15bd-4d3f-9f55-b458fa9270e7 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:00.405806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1339/2666] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:20:00 2022] POST /identity/v3/users => generated 285 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:20:00.410225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480bf5f-e7fb-4f0d-82ad-2afb1da76c08 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:00.422260 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480bf5f-e7fb-4f0d-82ad-2afb1da76c08 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:00.427960 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f420483-9567-46d0-9244-f3ed365d0359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:00.428647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f420483-9567-46d0-9244-f3ed365d0359 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:00.428932 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f420483-9567-46d0-9244-f3ed365d0359 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:00.429214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f420483-9567-46d0-9244-f3ed365d0359 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:00.429949 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f420483-9567-46d0-9244-f3ed365d0359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:00.431630 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f420483-9567-46d0-9244-f3ed365d0359 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:00.443456 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1328/2667] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:20:00.447810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b930021-15bd-4d3f-9f55-b458fa9270e7 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:00.458685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b930021-15bd-4d3f-9f55-b458fa9270e7 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:00.463298 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8c1b1-74a4-40b8-a3af-7f2f7c1fda59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:00.464181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8c1b1-74a4-40b8-a3af-7f2f7c1fda59 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:00.464465 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8c1b1-74a4-40b8-a3af-7f2f7c1fda59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:00.464743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8c1b1-74a4-40b8-a3af-7f2f7c1fda59 admin admin] PATH_INFO: `/v3/projects/639867d518274ffd82a8e539ea805e7d/users/f06f68a852f44ef1bb414955b9db4fef/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:00.480268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8c1b1-74a4-40b8-a3af-7f2f7c1fda59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=639867d518274ffd82a8e539ea805e7d, user_id=f06f68a852f44ef1bb414955b9db4fef, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:00.483001 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8c1b1-74a4-40b8-a3af-7f2f7c1fda59 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:00.566423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1340/2668] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:20:00 2022] PUT /identity/v3/projects/639867d518274ffd82a8e539ea805e7d/users/f06f68a852f44ef1bb414955b9db4fef/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:00.570634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f420483-9567-46d0-9244-f3ed365d0359 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:00.581520 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f420483-9567-46d0-9244-f3ed365d0359 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:00.638376 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c732b6-6c2e-4c6b-975b-e45b23950dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:00.639047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c732b6-6c2e-4c6b-975b-e45b23950dae admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:00.639335 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c732b6-6c2e-4c6b-975b-e45b23950dae admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:00.639612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c732b6-6c2e-4c6b-975b-e45b23950dae admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:00.640327 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c732b6-6c2e-4c6b-975b-e45b23950dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:00.641905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c732b6-6c2e-4c6b-975b-e45b23950dae admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:00.651805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1329/2669] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:20:00.656105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8c1b1-74a4-40b8-a3af-7f2f7c1fda59 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:00.666546 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8c1b1-74a4-40b8-a3af-7f2f7c1fda59 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:00.671233 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d57d6-af24-41f9-87b2-be013f74aa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:00.672110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d57d6-af24-41f9-87b2-be013f74aa1d admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:00.672398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d57d6-af24-41f9-87b2-be013f74aa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:00.672678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d57d6-af24-41f9-87b2-be013f74aa1d admin admin] PATH_INFO: `/v3/projects/639867d518274ffd82a8e539ea805e7d/users/f06f68a852f44ef1bb414955b9db4fef/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:00.675287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d57d6-af24-41f9-87b2-be013f74aa1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=639867d518274ffd82a8e539ea805e7d, user_id=f06f68a852f44ef1bb414955b9db4fef, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:00.677901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d57d6-af24-41f9-87b2-be013f74aa1d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:00.688146 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1341/2670] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:20:00 2022] PUT /identity/v3/projects/639867d518274ffd82a8e539ea805e7d/users/f06f68a852f44ef1bb414955b9db4fef/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:00.699122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1666a-538d-4bf9-b7cf-621783dec927 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:00.699418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1666a-538d-4bf9-b7cf-621783dec927 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:00.699704 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1666a-538d-4bf9-b7cf-621783dec927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:00.727416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-62d1666a-538d-4bf9-b7cf-621783dec927 None None] MFA Rules not processed for user `f06f68a852f44ef1bb414955b9db4fef`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:00.728633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-62d1666a-538d-4bf9-b7cf-621783dec927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:00.796715 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1330/2671] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:00 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:20:00.809201 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d57d6-af24-41f9-87b2-be013f74aa1d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:00.810999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5e0d57d6-af24-41f9-87b2-be013f74aa1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:00.874491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d57d6-af24-41f9-87b2-be013f74aa1d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:00.878836 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441b2d6-4432-40aa-8329-570082197392 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:00.879490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441b2d6-4432-40aa-8329-570082197392 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:00.879776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441b2d6-4432-40aa-8329-570082197392 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:00.880054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441b2d6-4432-40aa-8329-570082197392 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:00.889244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441b2d6-4432-40aa-8329-570082197392 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:00.891503 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441b2d6-4432-40aa-8329-570082197392 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:00.900565 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1342/2672] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:20:00 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:20:01.116591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1666a-538d-4bf9-b7cf-621783dec927 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:01.127157 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1666a-538d-4bf9-b7cf-621783dec927 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:01.130785 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7a826-dce2-45ff-bf02-265968424ed7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:01.134431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7a826-dce2-45ff-bf02-265968424ed7 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:01.134431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7a826-dce2-45ff-bf02-265968424ed7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:01.134431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7a826-dce2-45ff-bf02-265968424ed7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:01.193270 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7a826-dce2-45ff-bf02-265968424ed7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:01.194757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7a826-dce2-45ff-bf02-265968424ed7 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:01.203994 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1331/2673] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:20:01 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:20:10.895988 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:11.199830 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=96 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:14.930367 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441b2d6-4432-40aa-8329-570082197392 service cinder] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:14.931913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5441b2d6-4432-40aa-8329-570082197392 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:15.045087 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441b2d6-4432-40aa-8329-570082197392 service cinder] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:15.049472 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0772f-1235-4c2a-b8f2-480439ed151c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:15.050122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0772f-1235-4c2a-b8f2-480439ed151c service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:15.050441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0772f-1235-4c2a-b8f2-480439ed151c service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:15.050719 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0772f-1235-4c2a-b8f2-480439ed151c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:15.052565 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8ee0772f-1235-4c2a-b8f2-480439ed151c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:15.062311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0772f-1235-4c2a-b8f2-480439ed151c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:15.063766 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0772f-1235-4c2a-b8f2-480439ed151c service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:15.072008 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1343/2674] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:20:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:20:21.337114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7a826-dce2-45ff-bf02-265968424ed7 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:21.447232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7a826-dce2-45ff-bf02-265968424ed7 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:21.451814 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:21.452718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:21.453003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:21.453283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] PATH_INFO: `/v3/users/baead5cd60bf4471a55a705347b13460` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:21.455140 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baead5cd60bf4471a55a705347b13460)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:21.457162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:21.662762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baead5cd60bf4471a55a705347b13460'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:21.663425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baead5cd60bf4471a55a705347b13460'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:21.664045 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baead5cd60bf4471a55a705347b13460'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:21.742087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baead5cd60bf4471a55a705347b13460'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:21.756374 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baead5cd60bf4471a55a705347b13460'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:21.769282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1332/2675] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:20:21 2022] DELETE /identity/v3/users/baead5cd60bf4471a55a705347b13460 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:21.773762 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f41d2-eba2-403f-b94f-36748b97b91d None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:21.774235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f41d2-eba2-403f-b94f-36748b97b91d None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:21.774398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f41d2-eba2-403f-b94f-36748b97b91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:21.814675 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-3d3f41d2-eba2-403f-b94f-36748b97b91d None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:21.815939 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3d3f41d2-eba2-403f-b94f-36748b97b91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:21.875739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1344/2676] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:21.885983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:21.887484 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:21.931166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:21.946242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd26ef5-25d4-4d20-8aa3-ad22441f6692 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:21.949840 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623dd51-fa71-44b1-b8af-ff1918173240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:21.950524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623dd51-fa71-44b1-b8af-ff1918173240 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:21.950812 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623dd51-fa71-44b1-b8af-ff1918173240 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:21.951089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623dd51-fa71-44b1-b8af-ff1918173240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:21.960797 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623dd51-fa71-44b1-b8af-ff1918173240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:21.962215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623dd51-fa71-44b1-b8af-ff1918173240 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:21.972247 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1333/2677] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:20:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:20:22.627219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f41d2-eba2-403f-b94f-36748b97b91d None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:22.628658 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3d3f41d2-eba2-403f-b94f-36748b97b91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:22.641050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f41d2-eba2-403f-b94f-36748b97b91d None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:22.646904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:22.647801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:22.648246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:22.648531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] PATH_INFO: `/v3/projects/d99194853de64e88985402c874bf90dd` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:22.649894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d99194853de64e88985402c874bf90dd)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:22.651868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:22.782341 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:22.795015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd99194853de64e88985402c874bf90dd', 'request_id': 'req-e58ac622-f5de-4dea-b62b-75135ed61115'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:22.801770 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd99194853de64e88985402c874bf90dd', 'request_id': 'req-e58ac622-f5de-4dea-b62b-75135ed61115'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:22.807510 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd99194853de64e88985402c874bf90dd', 'request_id': 'req-e58ac622-f5de-4dea-b62b-75135ed61115'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:22.870846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] The token cache is being invalidate because project d99194853de64e88985402c874bf90dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:20:22.871121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:22.871741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:22.873395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1345/2678] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:20:22 2022] DELETE /identity/v3/projects/d99194853de64e88985402c874bf90dd => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:22.878229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623dd51-fa71-44b1-b8af-ff1918173240 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:22.888252 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623dd51-fa71-44b1-b8af-ff1918173240 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:22.944424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85516497-c964-49e6-8f94-2fb565b3633a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:22.945328 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85516497-c964-49e6-8f94-2fb565b3633a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:22.945620 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85516497-c964-49e6-8f94-2fb565b3633a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:22.945904 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85516497-c964-49e6-8f94-2fb565b3633a admin admin] PATH_INFO: `/v3/users/f06f68a852f44ef1bb414955b9db4fef` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:22.947122 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85516497-c964-49e6-8f94-2fb565b3633a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f06f68a852f44ef1bb414955b9db4fef)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:22.949043 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85516497-c964-49e6-8f94-2fb565b3633a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:23.085000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-85516497-c964-49e6-8f94-2fb565b3633a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f06f68a852f44ef1bb414955b9db4fef'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:23.085755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-85516497-c964-49e6-8f94-2fb565b3633a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f06f68a852f44ef1bb414955b9db4fef'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:23.086457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-85516497-c964-49e6-8f94-2fb565b3633a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f06f68a852f44ef1bb414955b9db4fef'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:23.168849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-85516497-c964-49e6-8f94-2fb565b3633a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f06f68a852f44ef1bb414955b9db4fef'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:23.182228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-85516497-c964-49e6-8f94-2fb565b3633a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f06f68a852f44ef1bb414955b9db4fef'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:23.195767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1334/2679] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:20:22 2022] DELETE /identity/v3/users/f06f68a852f44ef1bb414955b9db4fef => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:20:23.604516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:23.604516 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:23.620905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ac622-f5de-4dea-b62b-75135ed61115 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:23.682131 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75415269-8e4f-4bda-82a4-f2f63f71143a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:23.683150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75415269-8e4f-4bda-82a4-f2f63f71143a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:23.683308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75415269-8e4f-4bda-82a4-f2f63f71143a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:23.683575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75415269-8e4f-4bda-82a4-f2f63f71143a admin admin] PATH_INFO: `/v3/projects/639867d518274ffd82a8e539ea805e7d` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:23.684802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75415269-8e4f-4bda-82a4-f2f63f71143a admin admin] RBAC: Authorizing `identity:delete_project(project_id=639867d518274ffd82a8e539ea805e7d)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:23.686755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75415269-8e4f-4bda-82a4-f2f63f71143a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:23.795328 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-75415269-8e4f-4bda-82a4-f2f63f71143a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '639867d518274ffd82a8e539ea805e7d', 'request_id': 'req-75415269-8e4f-4bda-82a4-f2f63f71143a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:23.801786 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-75415269-8e4f-4bda-82a4-f2f63f71143a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '639867d518274ffd82a8e539ea805e7d', 'request_id': 'req-75415269-8e4f-4bda-82a4-f2f63f71143a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:23.807501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-75415269-8e4f-4bda-82a4-f2f63f71143a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '639867d518274ffd82a8e539ea805e7d', 'request_id': 'req-75415269-8e4f-4bda-82a4-f2f63f71143a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:23.901371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-75415269-8e4f-4bda-82a4-f2f63f71143a admin admin] The token cache is being invalidate because project 639867d518274ffd82a8e539ea805e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:20:23.901589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-75415269-8e4f-4bda-82a4-f2f63f71143a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:23.902207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-75415269-8e4f-4bda-82a4-f2f63f71143a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:23.903820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1346/2680] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:20:23 2022] DELETE /identity/v3/projects/639867d518274ffd82a8e539ea805e7d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:20:23.909836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e90425-533f-49d4-809d-64dbae12d028 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:23.911903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e90425-533f-49d4-809d-64dbae12d028 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:23.911903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e90425-533f-49d4-809d-64dbae12d028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:23.930121 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b3e90425-533f-49d4-809d-64dbae12d028 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:23.931383 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b3e90425-533f-49d4-809d-64dbae12d028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:23.982218 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:23.990897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1335/2681] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:20:24.057910 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e529d-0d5f-4a04-b9dd-a87dc72a4677 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:24.058207 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e529d-0d5f-4a04-b9dd-a87dc72a4677 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:24.058533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e529d-0d5f-4a04-b9dd-a87dc72a4677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:24.079438 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-991e529d-0d5f-4a04-b9dd-a87dc72a4677 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:24.080766 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-991e529d-0d5f-4a04-b9dd-a87dc72a4677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:24.089410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1347/2682] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:20:24.093654 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e90425-533f-49d4-809d-64dbae12d028 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:24.105832 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e90425-533f-49d4-809d-64dbae12d028 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:24.111170 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6624a5-a0e5-489b-b84b-81140597a168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:24.112120 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6624a5-a0e5-489b-b84b-81140597a168 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:24.112410 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6624a5-a0e5-489b-b84b-81140597a168 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:24.112696 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6624a5-a0e5-489b-b84b-81140597a168 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:24.113440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6624a5-a0e5-489b-b84b-81140597a168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:20:24.113756 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6624a5-a0e5-489b-b84b-81140597a168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:24.115670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6624a5-a0e5-489b-b84b-81140597a168 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:24.127834 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1336/2683] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:20:24.132725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e529d-0d5f-4a04-b9dd-a87dc72a4677 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:24.142132 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e529d-0d5f-4a04-b9dd-a87dc72a4677 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:24.146712 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef635f-81a7-4910-bf2f-5aee4be75fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:24.147375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef635f-81a7-4910-bf2f-5aee4be75fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:24.147659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef635f-81a7-4910-bf2f-5aee4be75fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:24.147935 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef635f-81a7-4910-bf2f-5aee4be75fc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:24.148632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef635f-81a7-4910-bf2f-5aee4be75fc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:24.150536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef635f-81a7-4910-bf2f-5aee4be75fc3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:24.246306 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1348/2684] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:20:24 2022] POST /identity/v3/projects => generated 409 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:20:24.250634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6624a5-a0e5-489b-b84b-81140597a168 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:24.261062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6624a5-a0e5-489b-b84b-81140597a168 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:24.318039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81262c4c-ed00-4637-ad47-61dbedd7412a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:24.318784 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262c4c-ed00-4637-ad47-61dbedd7412a admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:24.319080 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262c4c-ed00-4637-ad47-61dbedd7412a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:24.319367 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262c4c-ed00-4637-ad47-61dbedd7412a admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:24.320076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81262c4c-ed00-4637-ad47-61dbedd7412a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:24.322008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81262c4c-ed00-4637-ad47-61dbedd7412a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:24.435163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1337/2685] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:20:24 2022] POST /identity/v3/users => generated 305 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:20:24.439926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef635f-81a7-4910-bf2f-5aee4be75fc3 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:24.450596 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef635f-81a7-4910-bf2f-5aee4be75fc3 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:24.455628 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8dce3-2c11-4835-ad6a-7f2bf2edd934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:24.456298 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8dce3-2c11-4835-ad6a-7f2bf2edd934 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:24.456585 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8dce3-2c11-4835-ad6a-7f2bf2edd934 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:24.456868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8dce3-2c11-4835-ad6a-7f2bf2edd934 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:24.457600 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8dce3-2c11-4835-ad6a-7f2bf2edd934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:24.459299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8dce3-2c11-4835-ad6a-7f2bf2edd934 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:24.469325 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1349/2686] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:20:24.473607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81262c4c-ed00-4637-ad47-61dbedd7412a admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:24.483818 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81262c4c-ed00-4637-ad47-61dbedd7412a admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:24.489028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cb0d2-ced3-4bef-8abc-6ecc5f0c8c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:24.489694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cb0d2-ced3-4bef-8abc-6ecc5f0c8c51 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:24.490239 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cb0d2-ced3-4bef-8abc-6ecc5f0c8c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:24.490566 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cb0d2-ced3-4bef-8abc-6ecc5f0c8c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:24.491296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698cb0d2-ced3-4bef-8abc-6ecc5f0c8c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:24.492894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698cb0d2-ced3-4bef-8abc-6ecc5f0c8c51 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:24.504306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1338/2687] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:20:24.508606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8dce3-2c11-4835-ad6a-7f2bf2edd934 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:24.519278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8dce3-2c11-4835-ad6a-7f2bf2edd934 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:24.523552 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b746da-cd6a-48fd-aeb2-cc7ac690df91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:24.524441 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b746da-cd6a-48fd-aeb2-cc7ac690df91 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:24.524733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b746da-cd6a-48fd-aeb2-cc7ac690df91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:24.525016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b746da-cd6a-48fd-aeb2-cc7ac690df91 admin admin] PATH_INFO: `/v3/projects/64b8f1588b8e481ab672314fc625da68/users/6d2578b407364fce9f8a6465eac95469/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:24.533720 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:24.544380 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b746da-cd6a-48fd-aeb2-cc7ac690df91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b8f1588b8e481ab672314fc625da68, user_id=6d2578b407364fce9f8a6465eac95469, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:24.547108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b746da-cd6a-48fd-aeb2-cc7ac690df91 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:24.645723 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1350/2688] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:20:24 2022] PUT /identity/v3/projects/64b8f1588b8e481ab672314fc625da68/users/6d2578b407364fce9f8a6465eac95469/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:24.656001 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5947ed3-4301-4217-bf12-ca39148f3a2c None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:24.656366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5947ed3-4301-4217-bf12-ca39148f3a2c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:24.656754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5947ed3-4301-4217-bf12-ca39148f3a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:24.687929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c5947ed3-4301-4217-bf12-ca39148f3a2c None None] MFA Rules not processed for user `6d2578b407364fce9f8a6465eac95469`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:24.687929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c5947ed3-4301-4217-bf12-ca39148f3a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:24.756500 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1339/2689] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:24 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:24.761173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b746da-cd6a-48fd-aeb2-cc7ac690df91 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:24.824114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b746da-cd6a-48fd-aeb2-cc7ac690df91 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:24.828810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352a8b0-fafd-4f92-b936-ca0eb9a52f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:24.830074 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352a8b0-fafd-4f92-b936-ca0eb9a52f07 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:24.830410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352a8b0-fafd-4f92-b936-ca0eb9a52f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:24.830687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352a8b0-fafd-4f92-b936-ca0eb9a52f07 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:24.831381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352a8b0-fafd-4f92-b936-ca0eb9a52f07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:24.833311 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352a8b0-fafd-4f92-b936-ca0eb9a52f07 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:24.921273 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1351/2690] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:20:24 2022] POST /identity/v3/projects => generated 409 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:20:24.925664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5947ed3-4301-4217-bf12-ca39148f3a2c None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:24.936799 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5947ed3-4301-4217-bf12-ca39148f3a2c None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:24.988996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:24.996324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ead30-72f2-4b3a-b836-56ee242bca95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:24.997010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ead30-72f2-4b3a-b836-56ee242bca95 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:24.997303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ead30-72f2-4b3a-b836-56ee242bca95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:24.997594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ead30-72f2-4b3a-b836-56ee242bca95 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:24.998351 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ead30-72f2-4b3a-b836-56ee242bca95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:25.000279 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ead30-72f2-4b3a-b836-56ee242bca95 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:25.068487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1340/2691] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:20:24 2022] POST /identity/v3/users => generated 311 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:20:25.072907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352a8b0-fafd-4f92-b936-ca0eb9a52f07 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:25.084242 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352a8b0-fafd-4f92-b936-ca0eb9a52f07 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:25.088936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209dabb2-b43c-422e-b2e8-57e00ce84ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:25.089602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dabb2-b43c-422e-b2e8-57e00ce84ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:25.089911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dabb2-b43c-422e-b2e8-57e00ce84ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:25.090173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dabb2-b43c-422e-b2e8-57e00ce84ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:25.090929 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209dabb2-b43c-422e-b2e8-57e00ce84ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:25.092524 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209dabb2-b43c-422e-b2e8-57e00ce84ea8 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:25.104052 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1352/2692] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:20:25.108284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ead30-72f2-4b3a-b836-56ee242bca95 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:25.119146 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ead30-72f2-4b3a-b836-56ee242bca95 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:25.123851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a10f5-e501-4780-994f-27ddadd941cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:25.124757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a10f5-e501-4780-994f-27ddadd941cb admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:25.125047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a10f5-e501-4780-994f-27ddadd941cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:25.125330 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a10f5-e501-4780-994f-27ddadd941cb admin admin] PATH_INFO: `/v3/projects/3261c2aa4e6b43caaf7198bdd3e3b0d2/users/5148ab3ee8b1451aacde915643b3231c/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:25.140569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a10f5-e501-4780-994f-27ddadd941cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3261c2aa4e6b43caaf7198bdd3e3b0d2, user_id=5148ab3ee8b1451aacde915643b3231c, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:25.143961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a10f5-e501-4780-994f-27ddadd941cb admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:25.223169 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1341/2693] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:20:25 2022] PUT /identity/v3/projects/3261c2aa4e6b43caaf7198bdd3e3b0d2/users/5148ab3ee8b1451aacde915643b3231c/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:25.227344 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209dabb2-b43c-422e-b2e8-57e00ce84ea8 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:25.237687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209dabb2-b43c-422e-b2e8-57e00ce84ea8 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:25.295706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20534e-5161-48c3-8d27-057da1d349b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:25.296281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20534e-5161-48c3-8d27-057da1d349b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:25.296597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20534e-5161-48c3-8d27-057da1d349b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:25.296908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20534e-5161-48c3-8d27-057da1d349b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:25.297645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20534e-5161-48c3-8d27-057da1d349b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:25.299830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20534e-5161-48c3-8d27-057da1d349b6 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:25.310252 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1353/2694] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:20:25.317171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a10f5-e501-4780-994f-27ddadd941cb admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:25.327871 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a10f5-e501-4780-994f-27ddadd941cb admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:25.332828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ee646-7cfc-4ccb-9ad5-965094729769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:25.333708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ee646-7cfc-4ccb-9ad5-965094729769 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:25.333997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ee646-7cfc-4ccb-9ad5-965094729769 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:25.335256 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ee646-7cfc-4ccb-9ad5-965094729769 admin admin] PATH_INFO: `/v3/projects/3261c2aa4e6b43caaf7198bdd3e3b0d2/users/5148ab3ee8b1451aacde915643b3231c/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:25.336325 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ee646-7cfc-4ccb-9ad5-965094729769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3261c2aa4e6b43caaf7198bdd3e3b0d2, user_id=5148ab3ee8b1451aacde915643b3231c, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:25.339024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ee646-7cfc-4ccb-9ad5-965094729769 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:25.352820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1342/2695] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:20:25 2022] PUT /identity/v3/projects/3261c2aa4e6b43caaf7198bdd3e3b0d2/users/5148ab3ee8b1451aacde915643b3231c/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:25.361284 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa26ee3-9836-4f9f-86c0-c2dfe3b2f1a5 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:25.361750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa26ee3-9836-4f9f-86c0-c2dfe3b2f1a5 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:25.362044 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa26ee3-9836-4f9f-86c0-c2dfe3b2f1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:25.389962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-3aa26ee3-9836-4f9f-86c0-c2dfe3b2f1a5 None None] MFA Rules not processed for user `5148ab3ee8b1451aacde915643b3231c`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:25.391242 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-3aa26ee3-9836-4f9f-86c0-c2dfe3b2f1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:25.438413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:25.462346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1354/2696] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:25 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:25.473999 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ee646-7cfc-4ccb-9ad5-965094729769 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:25.479444 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0c7ee646-7cfc-4ccb-9ad5-965094729769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:25.541850 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ee646-7cfc-4ccb-9ad5-965094729769 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:25.545511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77632842-b7e0-40de-a223-87733ee66a69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:25.546201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77632842-b7e0-40de-a223-87733ee66a69 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:25.546534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77632842-b7e0-40de-a223-87733ee66a69 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:25.546815 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77632842-b7e0-40de-a223-87733ee66a69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:25.556686 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77632842-b7e0-40de-a223-87733ee66a69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:25.558112 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77632842-b7e0-40de-a223-87733ee66a69 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:25.567860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1343/2697] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:20:25 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:20:25.950089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa26ee3-9836-4f9f-86c0-c2dfe3b2f1a5 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:25.960531 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa26ee3-9836-4f9f-86c0-c2dfe3b2f1a5 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:25.977782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f6c48-0dbb-415e-9d46-4052ca4514de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:25.977782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f6c48-0dbb-415e-9d46-4052ca4514de service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:25.977782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f6c48-0dbb-415e-9d46-4052ca4514de service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:25.977782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f6c48-0dbb-415e-9d46-4052ca4514de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:26.024896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f6c48-0dbb-415e-9d46-4052ca4514de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:26.026366 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f6c48-0dbb-415e-9d46-4052ca4514de service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:26.035522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1355/2698] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:20:25 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:20:35.563742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=49,keystone:INSERT=5 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:36.031256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=36 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:38.210345 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77632842-b7e0-40de-a223-87733ee66a69 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:38.280191 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77632842-b7e0-40de-a223-87733ee66a69 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:38.284872 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:38.285754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:38.286041 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:38.286322 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] PATH_INFO: `/v3/users/6d2578b407364fce9f8a6465eac95469` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:38.287844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d2578b407364fce9f8a6465eac95469)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:38.289828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:38.556571 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d2578b407364fce9f8a6465eac95469'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:38.557349 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d2578b407364fce9f8a6465eac95469'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:38.558054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d2578b407364fce9f8a6465eac95469'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:38.616441 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d2578b407364fce9f8a6465eac95469'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:38.636675 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d2578b407364fce9f8a6465eac95469'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:38.653852 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1344/2699] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:20:38 2022] DELETE /identity/v3/users/6d2578b407364fce9f8a6465eac95469 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:38.660362 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6b724-4655-4223-827b-04acda9c380f None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:38.660913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6b724-4655-4223-827b-04acda9c380f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:38.660913 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6b724-4655-4223-827b-04acda9c380f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:38.682976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ded6b724-4655-4223-827b-04acda9c380f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:38.684456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ded6b724-4655-4223-827b-04acda9c380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:38.766757 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1356/2700] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:20:38.776346 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:38.777791 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:38.840168 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566fabf-83a9-49c1-89b0-de07a1949882 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:38.843955 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb5110-25c8-44c2-ad0a-abb039c4c4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:38.845164 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb5110-25c8-44c2-ad0a-abb039c4c4cf service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:38.845451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb5110-25c8-44c2-ad0a-abb039c4c4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:38.845731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb5110-25c8-44c2-ad0a-abb039c4c4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:38.855460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb5110-25c8-44c2-ad0a-abb039c4c4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:38.856899 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb5110-25c8-44c2-ad0a-abb039c4c4cf service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:38.868642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1345/2701] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:20:39.287152 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6b724-4655-4223-827b-04acda9c380f None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:39.288597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ded6b724-4655-4223-827b-04acda9c380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:39.302037 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6b724-4655-4223-827b-04acda9c380f None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:39.306753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:39.307595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:39.307875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:39.308150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] PATH_INFO: `/v3/projects/64b8f1588b8e481ab672314fc625da68` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:39.309347 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64b8f1588b8e481ab672314fc625da68)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:39.311283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:39.416830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64b8f1588b8e481ab672314fc625da68', 'request_id': 'req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:39.423610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64b8f1588b8e481ab672314fc625da68', 'request_id': 'req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:39.429755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64b8f1588b8e481ab672314fc625da68', 'request_id': 'req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:39.524544 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] The token cache is being invalidate because project 64b8f1588b8e481ab672314fc625da68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:20:39.524837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:39.525610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:39.527217 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1357/2702] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:20:39 2022] DELETE /identity/v3/projects/64b8f1588b8e481ab672314fc625da68 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:39.531314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb5110-25c8-44c2-ad0a-abb039c4c4cf service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:39.541301 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb5110-25c8-44c2-ad0a-abb039c4c4cf service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:39.560099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:39.598314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13b2f4-40f3-493e-84b8-3bd8425baf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:39.599225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13b2f4-40f3-493e-84b8-3bd8425baf51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:39.599615 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13b2f4-40f3-493e-84b8-3bd8425baf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:39.599799 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13b2f4-40f3-493e-84b8-3bd8425baf51 admin admin] PATH_INFO: `/v3/users/5148ab3ee8b1451aacde915643b3231c` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:39.601138 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13b2f4-40f3-493e-84b8-3bd8425baf51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5148ab3ee8b1451aacde915643b3231c)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:39.603108 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13b2f4-40f3-493e-84b8-3bd8425baf51 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:39.766615 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ea13b2f4-40f3-493e-84b8-3bd8425baf51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5148ab3ee8b1451aacde915643b3231c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:39.767394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ea13b2f4-40f3-493e-84b8-3bd8425baf51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5148ab3ee8b1451aacde915643b3231c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:39.768024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ea13b2f4-40f3-493e-84b8-3bd8425baf51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5148ab3ee8b1451aacde915643b3231c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:39.845485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ea13b2f4-40f3-493e-84b8-3bd8425baf51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5148ab3ee8b1451aacde915643b3231c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:39.859160 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-ea13b2f4-40f3-493e-84b8-3bd8425baf51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5148ab3ee8b1451aacde915643b3231c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:39.878739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1346/2703] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:20:39 2022] DELETE /identity/v3/users/5148ab3ee8b1451aacde915643b3231c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:40.278351 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:40.279851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:40.293639 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286e3ec-73b7-4e08-ae97-2c67ccb94ec1 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:40.356820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:40.357723 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:40.358022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:40.358302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a admin admin] PATH_INFO: `/v3/projects/3261c2aa4e6b43caaf7198bdd3e3b0d2` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:40.359584 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3261c2aa4e6b43caaf7198bdd3e3b0d2)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:40.361789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:40.444316 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:40.461729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3261c2aa4e6b43caaf7198bdd3e3b0d2', 'request_id': 'req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:40.467885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3261c2aa4e6b43caaf7198bdd3e3b0d2', 'request_id': 'req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:40.473729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3261c2aa4e6b43caaf7198bdd3e3b0d2', 'request_id': 'req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:40.575624 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a admin admin] The token cache is being invalidate because project 3261c2aa4e6b43caaf7198bdd3e3b0d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:20:40.580772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:40.580772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-73ebee95-bf43-4af3-b1f8-0c867a6e0e6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:40.580772 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1358/2704] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:20:40 2022] DELETE /identity/v3/projects/3261c2aa4e6b43caaf7198bdd3e3b0d2 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:40.583146 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070f6ae-efb8-4e77-abc3-fc2e814e956c None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:40.583622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070f6ae-efb8-4e77-abc3-fc2e814e956c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:40.583729 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070f6ae-efb8-4e77-abc3-fc2e814e956c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:40.603890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c070f6ae-efb8-4e77-abc3-fc2e814e956c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:40.605139 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c070f6ae-efb8-4e77-abc3-fc2e814e956c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:40.667232 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1347/2705] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:40.675115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c01964-ef2a-4b5e-ab80-191367f86071 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:40.675404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c01964-ef2a-4b5e-ab80-191367f86071 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:40.675685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c01964-ef2a-4b5e-ab80-191367f86071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:40.695541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d1c01964-ef2a-4b5e-ab80-191367f86071 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:40.696754 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d1c01964-ef2a-4b5e-ab80-191367f86071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:40.705375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1359/2706] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:40.709700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070f6ae-efb8-4e77-abc3-fc2e814e956c None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:40.720056 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070f6ae-efb8-4e77-abc3-fc2e814e956c None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:40.724670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c73de-20e5-42f9-84f5-db303cdf5833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:40.725328 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c73de-20e5-42f9-84f5-db303cdf5833 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:40.725615 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c73de-20e5-42f9-84f5-db303cdf5833 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:40.725898 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c73de-20e5-42f9-84f5-db303cdf5833 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:40.726665 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c73de-20e5-42f9-84f5-db303cdf5833 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:20:40.726979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c73de-20e5-42f9-84f5-db303cdf5833 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:40.728570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c73de-20e5-42f9-84f5-db303cdf5833 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:40.741414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1348/2707] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:20:40.745724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c01964-ef2a-4b5e-ab80-191367f86071 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:40.755931 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c01964-ef2a-4b5e-ab80-191367f86071 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:40.760578 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d492ea-3f66-484e-bcff-43ae5eb1cecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:40.761234 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d492ea-3f66-484e-bcff-43ae5eb1cecb admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:40.761517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d492ea-3f66-484e-bcff-43ae5eb1cecb admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:40.761793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d492ea-3f66-484e-bcff-43ae5eb1cecb admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:40.762527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d492ea-3f66-484e-bcff-43ae5eb1cecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:40.764413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d492ea-3f66-484e-bcff-43ae5eb1cecb admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:40.874290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1360/2708] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:20:40 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:20:40.878958 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c73de-20e5-42f9-84f5-db303cdf5833 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:40.889394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c73de-20e5-42f9-84f5-db303cdf5833 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:40.916640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:40.948532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9587df1-55a3-4e0d-a004-667270809859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:40.949207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9587df1-55a3-4e0d-a004-667270809859 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:40.949500 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9587df1-55a3-4e0d-a004-667270809859 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:40.949803 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9587df1-55a3-4e0d-a004-667270809859 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:40.950539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9587df1-55a3-4e0d-a004-667270809859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:40.952452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9587df1-55a3-4e0d-a004-667270809859 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:41.024841 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1349/2709] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:20:40 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:20:41.029023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d492ea-3f66-484e-bcff-43ae5eb1cecb admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:41.039746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d492ea-3f66-484e-bcff-43ae5eb1cecb admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:41.044645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff7638-7b63-4b84-8545-962784d64676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:41.045320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff7638-7b63-4b84-8545-962784d64676 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:41.045609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff7638-7b63-4b84-8545-962784d64676 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:41.045892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff7638-7b63-4b84-8545-962784d64676 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:41.046688 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff7638-7b63-4b84-8545-962784d64676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:41.049095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff7638-7b63-4b84-8545-962784d64676 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:41.059244 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1361/2710] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:20:41.064126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9587df1-55a3-4e0d-a004-667270809859 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:41.075083 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9587df1-55a3-4e0d-a004-667270809859 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:41.081046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997a3d5-1a96-4bf8-bd10-4164cc99fa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:41.081735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997a3d5-1a96-4bf8-bd10-4164cc99fa0a admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:41.082030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997a3d5-1a96-4bf8-bd10-4164cc99fa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:41.082313 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997a3d5-1a96-4bf8-bd10-4164cc99fa0a admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:41.083111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997a3d5-1a96-4bf8-bd10-4164cc99fa0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:41.090649 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997a3d5-1a96-4bf8-bd10-4164cc99fa0a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:41.096132 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1350/2711] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:20:41.100456 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff7638-7b63-4b84-8545-962784d64676 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:41.112037 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff7638-7b63-4b84-8545-962784d64676 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:41.117341 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010726db-9dbb-4560-a1b3-bdc2222f672b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:41.118230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010726db-9dbb-4560-a1b3-bdc2222f672b admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:41.118542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010726db-9dbb-4560-a1b3-bdc2222f672b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:41.118819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010726db-9dbb-4560-a1b3-bdc2222f672b admin admin] PATH_INFO: `/v3/projects/4a4e404c29e249d6928d2df3ff9d93dc/users/328ba18b6cd9485ba338cc3300d94587/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:41.136126 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010726db-9dbb-4560-a1b3-bdc2222f672b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4e404c29e249d6928d2df3ff9d93dc, user_id=328ba18b6cd9485ba338cc3300d94587, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:41.138865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010726db-9dbb-4560-a1b3-bdc2222f672b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:41.222545 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1362/2712] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:20:41 2022] PUT /identity/v3/projects/4a4e404c29e249d6928d2df3ff9d93dc/users/328ba18b6cd9485ba338cc3300d94587/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:41.232944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e7804-b530-4703-9872-cdefe8dc7c0d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:41.233244 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e7804-b530-4703-9872-cdefe8dc7c0d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:41.233531 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e7804-b530-4703-9872-cdefe8dc7c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:41.262413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-9d5e7804-b530-4703-9872-cdefe8dc7c0d None None] MFA Rules not processed for user `328ba18b6cd9485ba338cc3300d94587`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:41.263630 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9d5e7804-b530-4703-9872-cdefe8dc7c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:41.329979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1351/2713] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:41.334533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010726db-9dbb-4560-a1b3-bdc2222f672b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:41.396908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010726db-9dbb-4560-a1b3-bdc2222f672b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:41.401604 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fad87c-7d50-45f3-8245-f8a7853cdab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:41.402267 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fad87c-7d50-45f3-8245-f8a7853cdab0 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:41.402588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fad87c-7d50-45f3-8245-f8a7853cdab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:41.402870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fad87c-7d50-45f3-8245-f8a7853cdab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:41.403573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fad87c-7d50-45f3-8245-f8a7853cdab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:41.405533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fad87c-7d50-45f3-8245-f8a7853cdab0 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:41.498090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1363/2714] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:20:41 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:20:41.502982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e7804-b530-4703-9872-cdefe8dc7c0d None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:41.513739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e7804-b530-4703-9872-cdefe8dc7c0d None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:41.571644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9643830-5247-4a0f-863e-b67aaac17664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:41.572443 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9643830-5247-4a0f-863e-b67aaac17664 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:41.572734 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9643830-5247-4a0f-863e-b67aaac17664 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:41.573015 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9643830-5247-4a0f-863e-b67aaac17664 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:41.573706 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9643830-5247-4a0f-863e-b67aaac17664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:41.575670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9643830-5247-4a0f-863e-b67aaac17664 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:41.674055 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1352/2715] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:20:41 2022] POST /identity/v3/users => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:20:41.678243 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fad87c-7d50-45f3-8245-f8a7853cdab0 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:41.689821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fad87c-7d50-45f3-8245-f8a7853cdab0 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:41.694830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb668c9-f918-41cf-80f8-7f4cacd05d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:41.695510 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb668c9-f918-41cf-80f8-7f4cacd05d9b admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:41.695799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb668c9-f918-41cf-80f8-7f4cacd05d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:41.696083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb668c9-f918-41cf-80f8-7f4cacd05d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:41.696813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb668c9-f918-41cf-80f8-7f4cacd05d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:41.698574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb668c9-f918-41cf-80f8-7f4cacd05d9b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:41.710296 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1364/2716] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:20:41.714519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9643830-5247-4a0f-863e-b67aaac17664 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:41.725401 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9643830-5247-4a0f-863e-b67aaac17664 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:41.730104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395ad21-b6f4-4c85-af95-8e3ba20cfaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:41.731030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395ad21-b6f4-4c85-af95-8e3ba20cfaeb admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:41.731314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395ad21-b6f4-4c85-af95-8e3ba20cfaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:41.731593 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395ad21-b6f4-4c85-af95-8e3ba20cfaeb admin admin] PATH_INFO: `/v3/projects/62a911ab1d4c46a9a9fc58ade0f8510c/users/bc94b6b4cf0641a780f279857c27492c/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:41.748569 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395ad21-b6f4-4c85-af95-8e3ba20cfaeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a911ab1d4c46a9a9fc58ade0f8510c, user_id=bc94b6b4cf0641a780f279857c27492c, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:41.751964 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395ad21-b6f4-4c85-af95-8e3ba20cfaeb admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:41.849496 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1353/2717] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:20:41 2022] PUT /identity/v3/projects/62a911ab1d4c46a9a9fc58ade0f8510c/users/bc94b6b4cf0641a780f279857c27492c/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:41.853481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb668c9-f918-41cf-80f8-7f4cacd05d9b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:41.864944 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb668c9-f918-41cf-80f8-7f4cacd05d9b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:41.885121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:41.921476 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47543557-a1b3-450f-a603-7207ecc9434b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:41.922148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47543557-a1b3-450f-a603-7207ecc9434b admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:41.922470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47543557-a1b3-450f-a603-7207ecc9434b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:41.923412 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47543557-a1b3-450f-a603-7207ecc9434b admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:41.929443 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47543557-a1b3-450f-a603-7207ecc9434b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:41.929443 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47543557-a1b3-450f-a603-7207ecc9434b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:41.935633 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1365/2718] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:20:41.940200 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395ad21-b6f4-4c85-af95-8e3ba20cfaeb admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:41.946419 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:41.955018 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395ad21-b6f4-4c85-af95-8e3ba20cfaeb admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:41.957045 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7dde0-0758-4573-b3eb-fa34d2fb2eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:41.957934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7dde0-0758-4573-b3eb-fa34d2fb2eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:41.958217 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7dde0-0758-4573-b3eb-fa34d2fb2eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:41.958524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7dde0-0758-4573-b3eb-fa34d2fb2eaa admin admin] PATH_INFO: `/v3/projects/62a911ab1d4c46a9a9fc58ade0f8510c/users/bc94b6b4cf0641a780f279857c27492c/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:41.960521 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7dde0-0758-4573-b3eb-fa34d2fb2eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a911ab1d4c46a9a9fc58ade0f8510c, user_id=bc94b6b4cf0641a780f279857c27492c, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:41.963334 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7dde0-0758-4573-b3eb-fa34d2fb2eaa admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:41.975581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1354/2719] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:20:41 2022] PUT /identity/v3/projects/62a911ab1d4c46a9a9fc58ade0f8510c/users/bc94b6b4cf0641a780f279857c27492c/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:20:41.987717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5a3d9-7e32-4af3-b5f5-92896304fc13 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:41.988010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5a3d9-7e32-4af3-b5f5-92896304fc13 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:41.988295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5a3d9-7e32-4af3-b5f5-92896304fc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:42.017270 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-1bf5a3d9-7e32-4af3-b5f5-92896304fc13 None None] MFA Rules not processed for user `bc94b6b4cf0641a780f279857c27492c`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:42.018536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1bf5a3d9-7e32-4af3-b5f5-92896304fc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:42.089571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1366/2720] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:20:42.101447 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7dde0-0758-4573-b3eb-fa34d2fb2eaa admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:42.103403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-78f7dde0-0758-4573-b3eb-fa34d2fb2eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:42.163156 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7dde0-0758-4573-b3eb-fa34d2fb2eaa admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:42.166822 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757ab62-71e0-44d2-8958-4c5765756916 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:42.167471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757ab62-71e0-44d2-8958-4c5765756916 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:42.167755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757ab62-71e0-44d2-8958-4c5765756916 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:42.168033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757ab62-71e0-44d2-8958-4c5765756916 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:42.177539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757ab62-71e0-44d2-8958-4c5765756916 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:42.179022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757ab62-71e0-44d2-8958-4c5765756916 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:42.189226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1355/2721] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:20:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:20:42.342776 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5a3d9-7e32-4af3-b5f5-92896304fc13 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:42.353240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5a3d9-7e32-4af3-b5f5-92896304fc13 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:42.356890 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c71b8-ef35-42ba-aaf7-46a1febd9adc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:42.357537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c71b8-ef35-42ba-aaf7-46a1febd9adc service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:42.357827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c71b8-ef35-42ba-aaf7-46a1febd9adc service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:42.358107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c71b8-ef35-42ba-aaf7-46a1febd9adc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:42.419042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c71b8-ef35-42ba-aaf7-46a1febd9adc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:42.420492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c71b8-ef35-42ba-aaf7-46a1febd9adc service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:42.433432 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1367/2722] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:20:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:20:42.925247 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757ab62-71e0-44d2-8958-4c5765756916 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:42.991779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757ab62-71e0-44d2-8958-4c5765756916 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:42.996497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:42.997468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:42.997838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:42.998128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] PATH_INFO: `/v3/users/328ba18b6cd9485ba338cc3300d94587` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:42.999379 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=328ba18b6cd9485ba338cc3300d94587)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:43.001991 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:43.208135 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '328ba18b6cd9485ba338cc3300d94587'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:43.208863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '328ba18b6cd9485ba338cc3300d94587'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:43.209633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '328ba18b6cd9485ba338cc3300d94587'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:43.296313 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '328ba18b6cd9485ba338cc3300d94587'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:43.310606 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '328ba18b6cd9485ba338cc3300d94587'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:43.327191 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1356/2723] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:20:42 2022] DELETE /identity/v3/users/328ba18b6cd9485ba338cc3300d94587 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:43.337412 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9b535-4425-4f03-9e9d-e9ba05d3f24a None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:43.337810 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9b535-4425-4f03-9e9d-e9ba05d3f24a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:43.338299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9b535-4425-4f03-9e9d-e9ba05d3f24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:43.365341 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-9ce9b535-4425-4f03-9e9d-e9ba05d3f24a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:43.367145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9ce9b535-4425-4f03-9e9d-e9ba05d3f24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:43.417119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:43.441691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1368/2724] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:43.455102 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:43.456656 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:43.474064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:43.533185 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9edf3-a9d4-4027-b6e6-5f66d2e4ba60 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:43.537034 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2420c-c531-4641-8e5a-a31fb1da7965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:43.537709 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2420c-c531-4641-8e5a-a31fb1da7965 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:43.538002 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2420c-c531-4641-8e5a-a31fb1da7965 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:43.538286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2420c-c531-4641-8e5a-a31fb1da7965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:43.549568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2420c-c531-4641-8e5a-a31fb1da7965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:43.551089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2420c-c531-4641-8e5a-a31fb1da7965 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:43.562258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1357/2725] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:20:44.024616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9b535-4425-4f03-9e9d-e9ba05d3f24a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:44.026191 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9ce9b535-4425-4f03-9e9d-e9ba05d3f24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:44.040350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9b535-4425-4f03-9e9d-e9ba05d3f24a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:44.050408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:44.050939 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:44.051232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:44.051512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] PATH_INFO: `/v3/projects/4a4e404c29e249d6928d2df3ff9d93dc` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:44.052838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a4e404c29e249d6928d2df3ff9d93dc)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:44.055027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:44.172028 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a4e404c29e249d6928d2df3ff9d93dc', 'request_id': 'req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:44.178971 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a4e404c29e249d6928d2df3ff9d93dc', 'request_id': 'req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:44.184666 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a4e404c29e249d6928d2df3ff9d93dc', 'request_id': 'req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:44.290621 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] The token cache is being invalidate because project 4a4e404c29e249d6928d2df3ff9d93dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:20:44.290959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:44.291625 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:44.293188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1369/2726] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:20:44 2022] DELETE /identity/v3/projects/4a4e404c29e249d6928d2df3ff9d93dc => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:44.298286 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2420c-c531-4641-8e5a-a31fb1da7965 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:44.308517 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2420c-c531-4641-8e5a-a31fb1da7965 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:44.368193 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37c7eb-fe0c-4290-9351-76b9a03d8be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:44.369718 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37c7eb-fe0c-4290-9351-76b9a03d8be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:44.370010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37c7eb-fe0c-4290-9351-76b9a03d8be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:44.370296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37c7eb-fe0c-4290-9351-76b9a03d8be9 admin admin] PATH_INFO: `/v3/users/bc94b6b4cf0641a780f279857c27492c` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:44.372180 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37c7eb-fe0c-4290-9351-76b9a03d8be9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc94b6b4cf0641a780f279857c27492c)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:44.374173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37c7eb-fe0c-4290-9351-76b9a03d8be9 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:44.595571 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-da37c7eb-fe0c-4290-9351-76b9a03d8be9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc94b6b4cf0641a780f279857c27492c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:44.596268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-da37c7eb-fe0c-4290-9351-76b9a03d8be9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc94b6b4cf0641a780f279857c27492c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:44.596885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-da37c7eb-fe0c-4290-9351-76b9a03d8be9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc94b6b4cf0641a780f279857c27492c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:44.659623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-da37c7eb-fe0c-4290-9351-76b9a03d8be9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc94b6b4cf0641a780f279857c27492c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:44.674577 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-da37c7eb-fe0c-4290-9351-76b9a03d8be9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc94b6b4cf0641a780f279857c27492c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:44.687627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1358/2727] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:20:44 2022] DELETE /identity/v3/users/bc94b6b4cf0641a780f279857c27492c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:45.312856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:45.315315 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:45.330847 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346019ae-f3ef-45e1-9ae5-56f9fb0c3273 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:45.416336 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51efae-a003-4f79-910a-3e77aaad85bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:45.417171 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51efae-a003-4f79-910a-3e77aaad85bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:45.417461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51efae-a003-4f79-910a-3e77aaad85bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:45.417746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51efae-a003-4f79-910a-3e77aaad85bd admin admin] PATH_INFO: `/v3/projects/62a911ab1d4c46a9a9fc58ade0f8510c` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:45.424095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51efae-a003-4f79-910a-3e77aaad85bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a911ab1d4c46a9a9fc58ade0f8510c)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:45.424095 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51efae-a003-4f79-910a-3e77aaad85bd admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:45.565855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-8d51efae-a003-4f79-910a-3e77aaad85bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a911ab1d4c46a9a9fc58ade0f8510c', 'request_id': 'req-8d51efae-a003-4f79-910a-3e77aaad85bd'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:45.589204 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-8d51efae-a003-4f79-910a-3e77aaad85bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a911ab1d4c46a9a9fc58ade0f8510c', 'request_id': 'req-8d51efae-a003-4f79-910a-3e77aaad85bd'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:45.638513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-8d51efae-a003-4f79-910a-3e77aaad85bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a911ab1d4c46a9a9fc58ade0f8510c', 'request_id': 'req-8d51efae-a003-4f79-910a-3e77aaad85bd'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:45.749101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:45.755432 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-8d51efae-a003-4f79-910a-3e77aaad85bd admin admin] The token cache is being invalidate because project 62a911ab1d4c46a9a9fc58ade0f8510c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:20:45.755729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-8d51efae-a003-4f79-910a-3e77aaad85bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:45.756397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-8d51efae-a003-4f79-910a-3e77aaad85bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:45.757962 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1370/2728] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:20:45 2022] DELETE /identity/v3/projects/62a911ab1d4c46a9a9fc58ade0f8510c => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:45.762957 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be88c8-4b7c-41d9-8b7b-44c153e42b30 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:45.763460 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be88c8-4b7c-41d9-8b7b-44c153e42b30 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:45.763586 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be88c8-4b7c-41d9-8b7b-44c153e42b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:45.784878 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-37be88c8-4b7c-41d9-8b7b-44c153e42b30 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:45.786130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-37be88c8-4b7c-41d9-8b7b-44c153e42b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:45.831932 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:45.862851 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1359/2729] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:45.872861 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02ce37-6f34-4cce-bf83-bef4f85f5ffc None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:45.873160 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02ce37-6f34-4cce-bf83-bef4f85f5ffc None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:45.873447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02ce37-6f34-4cce-bf83-bef4f85f5ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:45.895317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ad02ce37-6f34-4cce-bf83-bef4f85f5ffc None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:45.898800 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ad02ce37-6f34-4cce-bf83-bef4f85f5ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:45.904638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1371/2730] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:45.909217 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be88c8-4b7c-41d9-8b7b-44c153e42b30 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:45.919544 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be88c8-4b7c-41d9-8b7b-44c153e42b30 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:45.925005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612cd80-4dd2-4e5f-9554-71cbc5287564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:45.929877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612cd80-4dd2-4e5f-9554-71cbc5287564 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:45.929877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612cd80-4dd2-4e5f-9554-71cbc5287564 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:45.929877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612cd80-4dd2-4e5f-9554-71cbc5287564 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:45.929877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612cd80-4dd2-4e5f-9554-71cbc5287564 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:20:45.929877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612cd80-4dd2-4e5f-9554-71cbc5287564 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:45.930559 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612cd80-4dd2-4e5f-9554-71cbc5287564 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:45.953812 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1360/2731] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:20:45.958655 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02ce37-6f34-4cce-bf83-bef4f85f5ffc None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:45.968781 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02ce37-6f34-4cce-bf83-bef4f85f5ffc None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:45.974075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0fd1e-30f4-47ec-adad-6e1fe3614cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:45.974796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0fd1e-30f4-47ec-adad-6e1fe3614cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:45.975093 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0fd1e-30f4-47ec-adad-6e1fe3614cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:45.975371 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0fd1e-30f4-47ec-adad-6e1fe3614cc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:45.976079 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0fd1e-30f4-47ec-adad-6e1fe3614cc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:45.977976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0fd1e-30f4-47ec-adad-6e1fe3614cc3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:46.066256 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1372/2732] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:20:45 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:20:46.072120 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612cd80-4dd2-4e5f-9554-71cbc5287564 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:46.082098 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612cd80-4dd2-4e5f-9554-71cbc5287564 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:46.144342 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcda51-5cda-42de-a615-5504d31bb356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:46.144877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcda51-5cda-42de-a615-5504d31bb356 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:46.145251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcda51-5cda-42de-a615-5504d31bb356 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:46.145445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcda51-5cda-42de-a615-5504d31bb356 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:46.146145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcda51-5cda-42de-a615-5504d31bb356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:46.148291 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcda51-5cda-42de-a615-5504d31bb356 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:46.266810 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1361/2733] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:20:46 2022] POST /identity/v3/users => generated 286 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:20:46.271392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0fd1e-30f4-47ec-adad-6e1fe3614cc3 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:46.283410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0fd1e-30f4-47ec-adad-6e1fe3614cc3 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:46.288365 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c456c6-ee51-4cba-9b0c-2515b6044644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:46.289115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c456c6-ee51-4cba-9b0c-2515b6044644 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:46.289402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c456c6-ee51-4cba-9b0c-2515b6044644 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:46.289681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c456c6-ee51-4cba-9b0c-2515b6044644 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:46.290448 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c456c6-ee51-4cba-9b0c-2515b6044644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:46.292080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c456c6-ee51-4cba-9b0c-2515b6044644 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:46.307501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1373/2734] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:20:46.319027 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcda51-5cda-42de-a615-5504d31bb356 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:46.336581 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcda51-5cda-42de-a615-5504d31bb356 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:46.344204 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780c687-ee96-4fff-bdbb-50ab84fcab54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:46.346021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780c687-ee96-4fff-bdbb-50ab84fcab54 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:46.346319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780c687-ee96-4fff-bdbb-50ab84fcab54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:46.346634 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780c687-ee96-4fff-bdbb-50ab84fcab54 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:46.347907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780c687-ee96-4fff-bdbb-50ab84fcab54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:46.350453 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780c687-ee96-4fff-bdbb-50ab84fcab54 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:46.365394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1362/2735] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:20:46.370039 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c456c6-ee51-4cba-9b0c-2515b6044644 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:46.383017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c456c6-ee51-4cba-9b0c-2515b6044644 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:46.388015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09ec2d-1664-4218-9ea2-31e3fb7c8487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:46.388932 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09ec2d-1664-4218-9ea2-31e3fb7c8487 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:46.389211 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09ec2d-1664-4218-9ea2-31e3fb7c8487 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:46.389494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09ec2d-1664-4218-9ea2-31e3fb7c8487 admin admin] PATH_INFO: `/v3/projects/d9c59097fc2144ef8b762ff528b131d8/users/7287349c36754c9e9443229c39b9bff5/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:46.433091 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09ec2d-1664-4218-9ea2-31e3fb7c8487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c59097fc2144ef8b762ff528b131d8, user_id=7287349c36754c9e9443229c39b9bff5, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:46.435894 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09ec2d-1664-4218-9ea2-31e3fb7c8487 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:46.524288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1374/2736] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:20:46 2022] PUT /identity/v3/projects/d9c59097fc2144ef8b762ff528b131d8/users/7287349c36754c9e9443229c39b9bff5/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:20:46.536229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c7230-5358-4991-9266-4cb441105705 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:46.536524 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c7230-5358-4991-9266-4cb441105705 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:46.536808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c7230-5358-4991-9266-4cb441105705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:46.570296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-0a6c7230-5358-4991-9266-4cb441105705 None None] MFA Rules not processed for user `7287349c36754c9e9443229c39b9bff5`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:46.571488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0a6c7230-5358-4991-9266-4cb441105705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:46.662468 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1363/2737] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:20:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:46.675337 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09ec2d-1664-4218-9ea2-31e3fb7c8487 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:46.677196 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6e09ec2d-1664-4218-9ea2-31e3fb7c8487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:46.782563 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09ec2d-1664-4218-9ea2-31e3fb7c8487 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:46.786510 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47d955-47cb-4de7-aa31-02c3b2135625 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28954d67689a46ee975fdfc9139e41eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:46.787219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47d955-47cb-4de7-aa31-02c3b2135625 service glance] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:46.787509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47d955-47cb-4de7-aa31-02c3b2135625 service glance] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:46.787789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47d955-47cb-4de7-aa31-02c3b2135625 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:46.798031 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47d955-47cb-4de7-aa31-02c3b2135625 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:46.799575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47d955-47cb-4de7-aa31-02c3b2135625 service glance] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:46.809978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1375/2738] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:20:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:20:56.658907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:56.805149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=63,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:21:16.028621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c7230-5358-4991-9266-4cb441105705 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:21:16.030425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0a6c7230-5358-4991-9266-4cb441105705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:16.109913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c7230-5358-4991-9266-4cb441105705 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:21:16.113833 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1a5e4-2fa6-4ba9-8b02-822c60dfe50d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:21:16.115357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1a5e4-2fa6-4ba9-8b02-822c60dfe50d service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:21:16.115357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1a5e4-2fa6-4ba9-8b02-822c60dfe50d service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:21:16.118284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1a5e4-2fa6-4ba9-8b02-822c60dfe50d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:21:16.118284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d9d1a5e4-2fa6-4ba9-8b02-822c60dfe50d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:16.199215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1a5e4-2fa6-4ba9-8b02-822c60dfe50d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:21:16.200726 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1a5e4-2fa6-4ba9-8b02-822c60dfe50d service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:21:16.214271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1364/2739] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:21:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:21:26.130130 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47d955-47cb-4de7-aa31-02c3b2135625 service glance] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:21:26.197400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47d955-47cb-4de7-aa31-02c3b2135625 service glance] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:21:26.204360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:21:26.204360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:21:26.204360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:21:26.204360 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] PATH_INFO: `/v3/users/7287349c36754c9e9443229c39b9bff5` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:21:26.205588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7287349c36754c9e9443229c39b9bff5)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:21:26.207687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:21:26.208584 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=59 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:21:26.422460 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7287349c36754c9e9443229c39b9bff5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:21:26.423122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7287349c36754c9e9443229c39b9bff5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:21:26.430760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7287349c36754c9e9443229c39b9bff5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:21:26.444168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7287349c36754c9e9443229c39b9bff5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:21:26.504069 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7287349c36754c9e9443229c39b9bff5'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:21:26.511497 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1376/2740] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:21:26 2022] DELETE /identity/v3/users/7287349c36754c9e9443229c39b9bff5 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:21:26.516411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d3c16-df00-4167-bcd2-25dc9bf00ae8 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:21:26.516706 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d3c16-df00-4167-bcd2-25dc9bf00ae8 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:21:26.517019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d3c16-df00-4167-bcd2-25dc9bf00ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:21:26.536823 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-a46d3c16-df00-4167-bcd2-25dc9bf00ae8 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:21:26.538074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a46d3c16-df00-4167-bcd2-25dc9bf00ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:26.600396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1365/2741] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:21:26.611178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:21:26.612610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:26.679288 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506cd07-c3b7-404f-82d2-0b1341c769f1 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:21:26.682915 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276017ab-a455-454b-86aa-6ca020e2bc4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:21:26.683575 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276017ab-a455-454b-86aa-6ca020e2bc4b service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:21:26.683860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276017ab-a455-454b-86aa-6ca020e2bc4b service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:21:26.684137 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276017ab-a455-454b-86aa-6ca020e2bc4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:21:26.696232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276017ab-a455-454b-86aa-6ca020e2bc4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:21:26.697793 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276017ab-a455-454b-86aa-6ca020e2bc4b service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:21:26.708872 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1377/2742] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:21:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:21:27.137781 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d3c16-df00-4167-bcd2-25dc9bf00ae8 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:21:27.139333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a46d3c16-df00-4167-bcd2-25dc9bf00ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:27.152671 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d3c16-df00-4167-bcd2-25dc9bf00ae8 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:21:27.164086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:21:27.165017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:21:27.165310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:21:27.165594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] PATH_INFO: `/v3/projects/d9c59097fc2144ef8b762ff528b131d8` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:21:27.166849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9c59097fc2144ef8b762ff528b131d8)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:21:27.169047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:21:27.305365 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9c59097fc2144ef8b762ff528b131d8', 'request_id': 'req-18800983-ba01-4516-b3bc-26f650a598c4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:21:27.387507 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9c59097fc2144ef8b762ff528b131d8', 'request_id': 'req-18800983-ba01-4516-b3bc-26f650a598c4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:21:27.394165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9c59097fc2144ef8b762ff528b131d8', 'request_id': 'req-18800983-ba01-4516-b3bc-26f650a598c4'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:21:27.407044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] The token cache is being invalidate because project d9c59097fc2144ef8b762ff528b131d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:21:27.407360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:21:27.408012 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:21:27.409626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1366/2743] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:21:27 2022] DELETE /identity/v3/projects/d9c59097fc2144ef8b762ff528b131d8 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:21:35.895622 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653538e-75ba-4ea7-b7bc-debdf77f56c5 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:21:35.908725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653538e-75ba-4ea7-b7bc-debdf77f56c5 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:21:35.908725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653538e-75ba-4ea7-b7bc-debdf77f56c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:21:35.917451 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-5653538e-75ba-4ea7-b7bc-debdf77f56c5 None None] MFA Rules not processed for user `61d6214556954b778dc86a0d376a8d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:21:35.918738 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5653538e-75ba-4ea7-b7bc-debdf77f56c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:35.942881 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:21:35.979661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1378/2744] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:21:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:21:36.345864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:21:36.347418 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:36.409158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18800983-ba01-4516-b3bc-26f650a598c4 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:21:36.413713 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362a9f9-b3be-4fd0-a7af-737763d2b41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:21:36.414486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362a9f9-b3be-4fd0-a7af-737763d2b41d service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:21:36.414796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362a9f9-b3be-4fd0-a7af-737763d2b41d service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:21:36.415086 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362a9f9-b3be-4fd0-a7af-737763d2b41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:21:36.417417 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1362a9f9-b3be-4fd0-a7af-737763d2b41d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:36.428542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362a9f9-b3be-4fd0-a7af-737763d2b41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:21:36.430035 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362a9f9-b3be-4fd0-a7af-737763d2b41d service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:21:36.440547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1367/2745] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:21:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:21:37.005706 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653538e-75ba-4ea7-b7bc-debdf77f56c5 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:21:37.006548 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362a9f9-b3be-4fd0-a7af-737763d2b41d service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:21:37.007174 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-5653538e-75ba-4ea7-b7bc-debdf77f56c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:37.008195 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-1362a9f9-b3be-4fd0-a7af-737763d2b41d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:37.019416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362a9f9-b3be-4fd0-a7af-737763d2b41d service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:21:37.023502 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03ef0a-8282-4938-afe3-306b17ac1eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:21:37.024152 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03ef0a-8282-4938-afe3-306b17ac1eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:21:37.024444 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03ef0a-8282-4938-afe3-306b17ac1eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:21:37.024723 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03ef0a-8282-4938-afe3-306b17ac1eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:21:37.026613 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2a03ef0a-8282-4938-afe3-306b17ac1eb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:37.053376 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:21:37.086250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653538e-75ba-4ea7-b7bc-debdf77f56c5 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:21:37.091299 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fcc38-f476-4028-a2f9-5ad3220c03be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:21:37.091978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fcc38-f476-4028-a2f9-5ad3220c03be service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:21:37.092262 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fcc38-f476-4028-a2f9-5ad3220c03be service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:21:37.092541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fcc38-f476-4028-a2f9-5ad3220c03be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:21:37.095310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c49fcc38-f476-4028-a2f9-5ad3220c03be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:37.107064 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03ef0a-8282-4938-afe3-306b17ac1eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:21:37.131100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49fcc38-f476-4028-a2f9-5ad3220c03be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:21:37.131258 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03ef0a-8282-4938-afe3-306b17ac1eb4 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:21:37.131361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49fcc38-f476-4028-a2f9-5ad3220c03be service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:21:37.131361 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1379/2746] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:21:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:21:37.131522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1368/2747] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:21:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:21:41.244314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fcc38-f476-4028-a2f9-5ad3220c03be service placement] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:21:41.246098 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c49fcc38-f476-4028-a2f9-5ad3220c03be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:41.273806 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fcc38-f476-4028-a2f9-5ad3220c03be service placement] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:21:41.278319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b6ab3-7237-45ad-a086-1c799a63e8de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:21:41.279116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b6ab3-7237-45ad-a086-1c799a63e8de service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:21:41.279400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b6ab3-7237-45ad-a086-1c799a63e8de service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:21:41.279951 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b6ab3-7237-45ad-a086-1c799a63e8de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:21:41.282260 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-701b6ab3-7237-45ad-a086-1c799a63e8de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:41.302632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b6ab3-7237-45ad-a086-1c799a63e8de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:21:41.304856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b6ab3-7237-45ad-a086-1c799a63e8de service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:21:41.318794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1380/2748] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:21:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:21:47.209843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=22 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:21:48.354310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03ef0a-8282-4938-afe3-306b17ac1eb4 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:21:48.366129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-2a03ef0a-8282-4938-afe3-306b17ac1eb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:48.371198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03ef0a-8282-4938-afe3-306b17ac1eb4 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:21:48.377600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91269be-4852-454b-b39d-1f883411a3a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:21:48.378252 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91269be-4852-454b-b39d-1f883411a3a1 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:21:48.378565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91269be-4852-454b-b39d-1f883411a3a1 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:21:48.378843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91269be-4852-454b-b39d-1f883411a3a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:21:48.380662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b91269be-4852-454b-b39d-1f883411a3a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:21:48.394001 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91269be-4852-454b-b39d-1f883411a3a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:21:48.395617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91269be-4852-454b-b39d-1f883411a3a1 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:21:48.406762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1369/2749] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:21:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:21:51.313542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=53 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:21:58.401786 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=6 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:23:24.175902 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b6ab3-7237-45ad-a086-1c799a63e8de service nova] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:23:24.177595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-701b6ab3-7237-45ad-a086-1c799a63e8de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:23:24.215926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b6ab3-7237-45ad-a086-1c799a63e8de service nova] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:23:24.221463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d944f-84ad-4b14-8a09-2aa82dbdbc71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:23:24.222116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d944f-84ad-4b14-8a09-2aa82dbdbc71 service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:23:24.222429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d944f-84ad-4b14-8a09-2aa82dbdbc71 service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:23:24.222713 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d944f-84ad-4b14-8a09-2aa82dbdbc71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:23:24.224492 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8a5d944f-84ad-4b14-8a09-2aa82dbdbc71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:23:24.237579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d944f-84ad-4b14-8a09-2aa82dbdbc71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:23:24.239209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d944f-84ad-4b14-8a09-2aa82dbdbc71 service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:23:24.249945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1381/2750] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:23:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:23:34.245011 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=15 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:23:53.173062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74846b-2d32-4720-9d22-9c65eb17a478 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:23:53.177469 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74846b-2d32-4720-9d22-9c65eb17a478 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:23:53.177469 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74846b-2d32-4720-9d22-9c65eb17a478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:23:53.195341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-fb74846b-2d32-4720-9d22-9c65eb17a478 None None] MFA Rules not processed for user `61d6214556954b778dc86a0d376a8d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:23:53.196475 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-fb74846b-2d32-4720-9d22-9c65eb17a478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:23:53.205865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1370/2751] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:23:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:03.189742 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:08.927119 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d944f-84ad-4b14-8a09-2aa82dbdbc71 service placement] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:08.969514 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d944f-84ad-4b14-8a09-2aa82dbdbc71 service placement] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:08.976206 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e9c99-e39e-43e8-a08a-bfb45672d6c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:08.976892 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e9c99-e39e-43e8-a08a-bfb45672d6c4 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:08.977183 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e9c99-e39e-43e8-a08a-bfb45672d6c4 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:08.977464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e9c99-e39e-43e8-a08a-bfb45672d6c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:08.979707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1c8e9c99-e39e-43e8-a08a-bfb45672d6c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:08.992856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e9c99-e39e-43e8-a08a-bfb45672d6c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:08.994528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e9c99-e39e-43e8-a08a-bfb45672d6c4 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:09.008463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1382/2752] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:16.486214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74846b-2d32-4720-9d22-9c65eb17a478 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:16.504918 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74846b-2d32-4720-9d22-9c65eb17a478 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:16.512603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6aa42-1078-44fb-9ca0-b2f8180ab0bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:16.512603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6aa42-1078-44fb-9ca0-b2f8180ab0bd service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:16.512603 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6aa42-1078-44fb-9ca0-b2f8180ab0bd service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:16.513028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6aa42-1078-44fb-9ca0-b2f8180ab0bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:16.514776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-dbb6aa42-1078-44fb-9ca0-b2f8180ab0bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:16.545153 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6aa42-1078-44fb-9ca0-b2f8180ab0bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:16.547612 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6aa42-1078-44fb-9ca0-b2f8180ab0bd service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:16.560495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1371/2753] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:19.000659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=13 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:26.555103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=11 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:19.910320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e9c99-e39e-43e8-a08a-bfb45672d6c4 service nova] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:19.922669 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e9c99-e39e-43e8-a08a-bfb45672d6c4 service nova] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:19.931066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b2080-38eb-4a48-917e-e4e114107420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:19.931066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b2080-38eb-4a48-917e-e4e114107420 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:19.931066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b2080-38eb-4a48-917e-e4e114107420 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:19.931066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b2080-38eb-4a48-917e-e4e114107420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:19.931681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6b1b2080-38eb-4a48-917e-e4e114107420 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:19.941088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b2080-38eb-4a48-917e-e4e114107420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:19.942616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b2080-38eb-4a48-917e-e4e114107420 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:19.952369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1383/2754] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:25:29.948198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=6 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:05.007255 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670a12e-4f1f-4a5e-82e6-ad9efb86b64f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:05.007848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670a12e-4f1f-4a5e-82e6-ad9efb86b64f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:05.007848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670a12e-4f1f-4a5e-82e6-ad9efb86b64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:05.027954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-b670a12e-4f1f-4a5e-82e6-ad9efb86b64f None None] MFA Rules not processed for user `61d6214556954b778dc86a0d376a8d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:05.029176 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b670a12e-4f1f-4a5e-82e6-ad9efb86b64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:05.036887 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1372/2755] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:26:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:15.022562 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:17.625773 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b2080-38eb-4a48-917e-e4e114107420 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:17.635976 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b2080-38eb-4a48-917e-e4e114107420 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:17.640381 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd81a3e-1b2b-4f6b-8612-0fbf3029377b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:17.641757 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd81a3e-1b2b-4f6b-8612-0fbf3029377b service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:17.642134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd81a3e-1b2b-4f6b-8612-0fbf3029377b service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:17.642591 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd81a3e-1b2b-4f6b-8612-0fbf3029377b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:17.644858 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8bd81a3e-1b2b-4f6b-8612-0fbf3029377b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:17.661343 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd81a3e-1b2b-4f6b-8612-0fbf3029377b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:17.662863 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd81a3e-1b2b-4f6b-8612-0fbf3029377b service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:17.677524 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1384/2756] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:27.670230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=6 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:00.823394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670a12e-4f1f-4a5e-82e6-ad9efb86b64f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:00.825797 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b670a12e-4f1f-4a5e-82e6-ad9efb86b64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:00.894425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670a12e-4f1f-4a5e-82e6-ad9efb86b64f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:00.899457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae5926-50a5-469c-b81f-b70e5d7ee076 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:00.900125 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae5926-50a5-469c-b81f-b70e5d7ee076 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:00.900411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae5926-50a5-469c-b81f-b70e5d7ee076 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:00.900690 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae5926-50a5-469c-b81f-b70e5d7ee076 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:00.914037 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae5926-50a5-469c-b81f-b70e5d7ee076 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:00.915534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae5926-50a5-469c-b81f-b70e5d7ee076 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:00.927935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1373/2757] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:05.741878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd81a3e-1b2b-4f6b-8612-0fbf3029377b service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:05.755157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd81a3e-1b2b-4f6b-8612-0fbf3029377b service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:05.771733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a566e02-1d22-46b1-b274-1d2dd149b242 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:05.772398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a566e02-1d22-46b1-b274-1d2dd149b242 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:05.772753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a566e02-1d22-46b1-b274-1d2dd149b242 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:05.773042 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a566e02-1d22-46b1-b274-1d2dd149b242 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:05.785605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a566e02-1d22-46b1-b274-1d2dd149b242 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:05.787548 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a566e02-1d22-46b1-b274-1d2dd149b242 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:05.802291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1385/2758] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:10.922529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=26 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:12.655805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae5926-50a5-469c-b81f-b70e5d7ee076 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:12.671360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae5926-50a5-469c-b81f-b70e5d7ee076 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:12.676056 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da346911-516d-4aa4-8178-c9712e5e88bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:12.676736 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da346911-516d-4aa4-8178-c9712e5e88bf service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:12.677053 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da346911-516d-4aa4-8178-c9712e5e88bf service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:12.677338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da346911-516d-4aa4-8178-c9712e5e88bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:12.679221 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-da346911-516d-4aa4-8178-c9712e5e88bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:12.752451 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da346911-516d-4aa4-8178-c9712e5e88bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:12.753909 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da346911-516d-4aa4-8178-c9712e5e88bf service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:12.764604 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1374/2759] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:15.797268 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=6 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:16.457826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a566e02-1d22-46b1-b274-1d2dd149b242 service nova] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:16.469522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a566e02-1d22-46b1-b274-1d2dd149b242 service nova] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:16.474931 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302dfdff-e977-4b70-8350-a5e6ff3cf107 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:16.475612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302dfdff-e977-4b70-8350-a5e6ff3cf107 service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:16.475868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302dfdff-e977-4b70-8350-a5e6ff3cf107 service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:16.476149 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302dfdff-e977-4b70-8350-a5e6ff3cf107 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:16.478426 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-302dfdff-e977-4b70-8350-a5e6ff3cf107 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:16.492665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302dfdff-e977-4b70-8350-a5e6ff3cf107 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:16.494289 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302dfdff-e977-4b70-8350-a5e6ff3cf107 service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:16.507020 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1386/2760] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:16.585740 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da346911-516d-4aa4-8178-c9712e5e88bf service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:16.587301 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-da346911-516d-4aa4-8178-c9712e5e88bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:16.602805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da346911-516d-4aa4-8178-c9712e5e88bf service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:16.611295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc2050-d609-44de-a87d-289c33bb938d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:16.612345 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc2050-d609-44de-a87d-289c33bb938d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:16.612864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc2050-d609-44de-a87d-289c33bb938d admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:16.612864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc2050-d609-44de-a87d-289c33bb938d admin admin] PATH_INFO: `/v3/users/61d6214556954b778dc86a0d376a8d6e` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:16.613883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc2050-d609-44de-a87d-289c33bb938d admin admin] RBAC: Authorizing `identity:delete_user(user_id=61d6214556954b778dc86a0d376a8d6e)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:16.615888 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc2050-d609-44de-a87d-289c33bb938d admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:16.810198 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1ccc2050-d609-44de-a87d-289c33bb938d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61d6214556954b778dc86a0d376a8d6e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:16.810854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1ccc2050-d609-44de-a87d-289c33bb938d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61d6214556954b778dc86a0d376a8d6e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:16.811466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1ccc2050-d609-44de-a87d-289c33bb938d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61d6214556954b778dc86a0d376a8d6e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:16.899158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1ccc2050-d609-44de-a87d-289c33bb938d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61d6214556954b778dc86a0d376a8d6e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:16.922751 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-1ccc2050-d609-44de-a87d-289c33bb938d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61d6214556954b778dc86a0d376a8d6e'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:16.937561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1375/2761] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:28:16 2022] DELETE /identity/v3/users/61d6214556954b778dc86a0d376a8d6e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:17.278509 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302dfdff-e977-4b70-8350-a5e6ff3cf107 service placement] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:17.292665 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302dfdff-e977-4b70-8350-a5e6ff3cf107 service placement] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:17.367349 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337773f4-30da-4a30-bcb7-b2847879d866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:17.368215 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337773f4-30da-4a30-bcb7-b2847879d866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:17.368499 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337773f4-30da-4a30-bcb7-b2847879d866 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:17.368782 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337773f4-30da-4a30-bcb7-b2847879d866 admin admin] PATH_INFO: `/v3/projects/c283d74bc1404c078fa9c350c8acfe4a` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:17.370016 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337773f4-30da-4a30-bcb7-b2847879d866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c283d74bc1404c078fa9c350c8acfe4a)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:17.372235 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337773f4-30da-4a30-bcb7-b2847879d866 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:17.518595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-337773f4-30da-4a30-bcb7-b2847879d866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c283d74bc1404c078fa9c350c8acfe4a', 'request_id': 'req-337773f4-30da-4a30-bcb7-b2847879d866'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:17.525466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-337773f4-30da-4a30-bcb7-b2847879d866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c283d74bc1404c078fa9c350c8acfe4a', 'request_id': 'req-337773f4-30da-4a30-bcb7-b2847879d866'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:17.532507 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-337773f4-30da-4a30-bcb7-b2847879d866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c283d74bc1404c078fa9c350c8acfe4a', 'request_id': 'req-337773f4-30da-4a30-bcb7-b2847879d866'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:17.629382 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-337773f4-30da-4a30-bcb7-b2847879d866 admin admin] The token cache is being invalidate because project c283d74bc1404c078fa9c350c8acfe4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:28:17.629612 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-337773f4-30da-4a30-bcb7-b2847879d866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:17.630219 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-337773f4-30da-4a30-bcb7-b2847879d866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:17.631860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1387/2762] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:28:17 2022] DELETE /identity/v3/projects/c283d74bc1404c078fa9c350c8acfe4a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:17.637993 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c806d-be44-4992-9e2a-0aadf483b7b2 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:17.638580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c806d-be44-4992-9e2a-0aadf483b7b2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:17.638580 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c806d-be44-4992-9e2a-0aadf483b7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:17.694546 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-491c806d-be44-4992-9e2a-0aadf483b7b2 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:17.695790 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-491c806d-be44-4992-9e2a-0aadf483b7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:17.737752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:17.763716 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1376/2763] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:17.772163 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8c3b9-be3f-4c88-a3fe-ff04b53efeba None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:17.772450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8c3b9-be3f-4c88-a3fe-ff04b53efeba None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:17.772730 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8c3b9-be3f-4c88-a3fe-ff04b53efeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:17.794627 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-6be8c3b9-be3f-4c88-a3fe-ff04b53efeba None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:17.795866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6be8c3b9-be3f-4c88-a3fe-ff04b53efeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:17.804302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1388/2764] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:17.809773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c806d-be44-4992-9e2a-0aadf483b7b2 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:17.826077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c806d-be44-4992-9e2a-0aadf483b7b2 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:17.833930 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55714b-ce22-4c2e-ba41-b0550c9fca8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:17.835079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55714b-ce22-4c2e-ba41-b0550c9fca8f admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:17.835565 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55714b-ce22-4c2e-ba41-b0550c9fca8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:17.836007 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55714b-ce22-4c2e-ba41-b0550c9fca8f admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:17.837263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55714b-ce22-4c2e-ba41-b0550c9fca8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:28:17.837806 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55714b-ce22-4c2e-ba41-b0550c9fca8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:17.841119 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55714b-ce22-4c2e-ba41-b0550c9fca8f admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:17.863422 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1377/2765] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:28:17.868090 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8c3b9-be3f-4c88-a3fe-ff04b53efeba None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:17.878342 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8c3b9-be3f-4c88-a3fe-ff04b53efeba None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:17.883308 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51e0fe-e95b-43ab-a725-f387bbdce9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:17.883920 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51e0fe-e95b-43ab-a725-f387bbdce9a0 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:17.884206 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51e0fe-e95b-43ab-a725-f387bbdce9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:17.884491 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51e0fe-e95b-43ab-a725-f387bbdce9a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:17.885213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51e0fe-e95b-43ab-a725-f387bbdce9a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:17.887309 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51e0fe-e95b-43ab-a725-f387bbdce9a0 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:17.992350 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1389/2766] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:28:17 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:28:17.998515 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55714b-ce22-4c2e-ba41-b0550c9fca8f admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:18.009749 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55714b-ce22-4c2e-ba41-b0550c9fca8f admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:18.071274 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202234b-05c1-42b9-922d-891ebea8de50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:18.071967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202234b-05c1-42b9-922d-891ebea8de50 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:18.072420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202234b-05c1-42b9-922d-891ebea8de50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:18.072547 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202234b-05c1-42b9-922d-891ebea8de50 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:18.073353 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202234b-05c1-42b9-922d-891ebea8de50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:18.075483 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202234b-05c1-42b9-922d-891ebea8de50 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:18.179576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1378/2767] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:28:17 2022] POST /identity/v3/users => generated 283 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:28:18.185213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51e0fe-e95b-43ab-a725-f387bbdce9a0 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:18.197409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51e0fe-e95b-43ab-a725-f387bbdce9a0 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:18.204317 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc12bc9-879d-4a6b-b767-caf55a4d01bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:18.205023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc12bc9-879d-4a6b-b767-caf55a4d01bb admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:18.205395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc12bc9-879d-4a6b-b767-caf55a4d01bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:18.205739 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc12bc9-879d-4a6b-b767-caf55a4d01bb admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:18.206533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc12bc9-879d-4a6b-b767-caf55a4d01bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:18.208173 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc12bc9-879d-4a6b-b767-caf55a4d01bb admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:18.219935 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1390/2768] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:28:18.224272 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202234b-05c1-42b9-922d-891ebea8de50 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:18.236125 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202234b-05c1-42b9-922d-891ebea8de50 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:18.241111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac92ab9-03f2-43d8-b8ae-627243a7f079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:18.241809 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac92ab9-03f2-43d8-b8ae-627243a7f079 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:18.242127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac92ab9-03f2-43d8-b8ae-627243a7f079 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:18.242442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac92ab9-03f2-43d8-b8ae-627243a7f079 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:18.243181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac92ab9-03f2-43d8-b8ae-627243a7f079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:18.244860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac92ab9-03f2-43d8-b8ae-627243a7f079 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:18.257678 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1379/2769] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:28:18.262230 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc12bc9-879d-4a6b-b767-caf55a4d01bb admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:18.273165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc12bc9-879d-4a6b-b767-caf55a4d01bb admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:18.278458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea95d8-65d8-4a29-9e6b-a3f0d99ce0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:18.279364 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea95d8-65d8-4a29-9e6b-a3f0d99ce0cb admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:18.279959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea95d8-65d8-4a29-9e6b-a3f0d99ce0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:18.279959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea95d8-65d8-4a29-9e6b-a3f0d99ce0cb admin admin] PATH_INFO: `/v3/projects/e74083c33c984f33bf55caa684b3fab7/users/48d16bf316434f829358ffaafd217328/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:18.297432 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea95d8-65d8-4a29-9e6b-a3f0d99ce0cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e74083c33c984f33bf55caa684b3fab7, user_id=48d16bf316434f829358ffaafd217328, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:18.300218 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea95d8-65d8-4a29-9e6b-a3f0d99ce0cb admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:18.367399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1391/2770] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:28:18 2022] PUT /identity/v3/projects/e74083c33c984f33bf55caa684b3fab7/users/48d16bf316434f829358ffaafd217328/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:18.384908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e17d0-4ab6-4f3e-afb7-5bbdd3ed5fd0 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:18.390214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e17d0-4ab6-4f3e-afb7-5bbdd3ed5fd0 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:18.390214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e17d0-4ab6-4f3e-afb7-5bbdd3ed5fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:18.422796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-cd8e17d0-4ab6-4f3e-afb7-5bbdd3ed5fd0 None None] MFA Rules not processed for user `48d16bf316434f829358ffaafd217328`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:18.423975 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-cd8e17d0-4ab6-4f3e-afb7-5bbdd3ed5fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:18.499439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1380/2771] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:18.504255 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea95d8-65d8-4a29-9e6b-a3f0d99ce0cb admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:18.510059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:18.575801 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea95d8-65d8-4a29-9e6b-a3f0d99ce0cb admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:18.580851 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b93d53-0924-4032-9888-d99416104db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:18.581574 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b93d53-0924-4032-9888-d99416104db3 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:18.581864 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b93d53-0924-4032-9888-d99416104db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:18.582151 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b93d53-0924-4032-9888-d99416104db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:18.582989 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b93d53-0924-4032-9888-d99416104db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:18.585495 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b93d53-0924-4032-9888-d99416104db3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:18.712636 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1392/2772] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:28:18 2022] POST /identity/v3/projects => generated 365 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:28:18.717471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e17d0-4ab6-4f3e-afb7-5bbdd3ed5fd0 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:18.729939 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e17d0-4ab6-4f3e-afb7-5bbdd3ed5fd0 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:18.767755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:18.792839 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7aebc-8596-4cd2-8242-3129aff5db44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:18.794245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7aebc-8596-4cd2-8242-3129aff5db44 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:18.794245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7aebc-8596-4cd2-8242-3129aff5db44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:18.794245 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7aebc-8596-4cd2-8242-3129aff5db44 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:18.794843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7aebc-8596-4cd2-8242-3129aff5db44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:18.796759 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7aebc-8596-4cd2-8242-3129aff5db44 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:18.882387 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1381/2773] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:28:18 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:28:18.886728 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b93d53-0924-4032-9888-d99416104db3 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:18.897214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b93d53-0924-4032-9888-d99416104db3 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:18.902591 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5af3d8-8210-44e1-b46d-6f16f33f45db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:18.903269 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5af3d8-8210-44e1-b46d-6f16f33f45db admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:18.903597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5af3d8-8210-44e1-b46d-6f16f33f45db admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:18.903882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5af3d8-8210-44e1-b46d-6f16f33f45db admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:18.904646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5af3d8-8210-44e1-b46d-6f16f33f45db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:18.906383 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5af3d8-8210-44e1-b46d-6f16f33f45db admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:18.918161 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1393/2774] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:28:18.922819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7aebc-8596-4cd2-8242-3129aff5db44 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:18.933542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7aebc-8596-4cd2-8242-3129aff5db44 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:18.938527 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe071b-6bff-42db-8c6c-fde3d7a7772c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:18.939448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe071b-6bff-42db-8c6c-fde3d7a7772c admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:18.939733 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe071b-6bff-42db-8c6c-fde3d7a7772c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:18.940008 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe071b-6bff-42db-8c6c-fde3d7a7772c admin admin] PATH_INFO: `/v3/projects/8cdfc81dd6024ea99d8d7a889675eea3/users/ffc6373250d6461393155dc28a29af89/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:18.956867 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe071b-6bff-42db-8c6c-fde3d7a7772c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cdfc81dd6024ea99d8d7a889675eea3, user_id=ffc6373250d6461393155dc28a29af89, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:18.959677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe071b-6bff-42db-8c6c-fde3d7a7772c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:19.033073 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1382/2775] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:28:18 2022] PUT /identity/v3/projects/8cdfc81dd6024ea99d8d7a889675eea3/users/ffc6373250d6461393155dc28a29af89/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:28:19.037746 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5af3d8-8210-44e1-b46d-6f16f33f45db admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:19.055997 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5af3d8-8210-44e1-b46d-6f16f33f45db admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:19.119427 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a63ff-b145-4b53-98c8-6ac268f0809a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:19.120656 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a63ff-b145-4b53-98c8-6ac268f0809a admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:19.120945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a63ff-b145-4b53-98c8-6ac268f0809a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:19.121220 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a63ff-b145-4b53-98c8-6ac268f0809a admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:19.121950 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a63ff-b145-4b53-98c8-6ac268f0809a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:19.123588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a63ff-b145-4b53-98c8-6ac268f0809a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:19.134043 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1394/2776] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:28:19.139382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe071b-6bff-42db-8c6c-fde3d7a7772c admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:19.150370 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe071b-6bff-42db-8c6c-fde3d7a7772c admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:19.155769 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682811a-d76a-46cb-85d9-c6bd8e6063a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:19.156714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682811a-d76a-46cb-85d9-c6bd8e6063a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:19.157009 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682811a-d76a-46cb-85d9-c6bd8e6063a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:19.157290 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682811a-d76a-46cb-85d9-c6bd8e6063a7 admin admin] PATH_INFO: `/v3/projects/8cdfc81dd6024ea99d8d7a889675eea3/users/ffc6373250d6461393155dc28a29af89/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:19.159440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682811a-d76a-46cb-85d9-c6bd8e6063a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cdfc81dd6024ea99d8d7a889675eea3, user_id=ffc6373250d6461393155dc28a29af89, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:19.162503 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682811a-d76a-46cb-85d9-c6bd8e6063a7 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:19.174003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1383/2777] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:28:19 2022] PUT /identity/v3/projects/8cdfc81dd6024ea99d8d7a889675eea3/users/ffc6373250d6461393155dc28a29af89/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:19.186684 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed2356-ac1f-483f-a978-fbdccc400a9b None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:19.186978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed2356-ac1f-483f-a978-fbdccc400a9b None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:19.187259 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed2356-ac1f-483f-a978-fbdccc400a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:19.217536 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-07ed2356-ac1f-483f-a978-fbdccc400a9b None None] MFA Rules not processed for user `ffc6373250d6461393155dc28a29af89`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:19.218817 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-07ed2356-ac1f-483f-a978-fbdccc400a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:19.289018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:19.293158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1395/2778] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:19.305938 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682811a-d76a-46cb-85d9-c6bd8e6063a7 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:19.306789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0682811a-d76a-46cb-85d9-c6bd8e6063a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:19.369413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682811a-d76a-46cb-85d9-c6bd8e6063a7 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:19.374113 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c856f8f-b13b-49b1-9150-29fa80a0d1e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:19.374831 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c856f8f-b13b-49b1-9150-29fa80a0d1e4 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:19.375126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c856f8f-b13b-49b1-9150-29fa80a0d1e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:19.375413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c856f8f-b13b-49b1-9150-29fa80a0d1e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:19.386425 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c856f8f-b13b-49b1-9150-29fa80a0d1e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:19.387922 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c856f8f-b13b-49b1-9150-29fa80a0d1e4 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:19.398373 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1384/2779] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:19.635846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed2356-ac1f-483f-a978-fbdccc400a9b None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:19.647825 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed2356-ac1f-483f-a978-fbdccc400a9b None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:19.654609 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2e670-403d-4d16-86fb-c5a5f159ac35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:19.655301 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2e670-403d-4d16-86fb-c5a5f159ac35 service cinder] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:19.655589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2e670-403d-4d16-86fb-c5a5f159ac35 service cinder] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:19.655874 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2e670-403d-4d16-86fb-c5a5f159ac35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:19.722216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2e670-403d-4d16-86fb-c5a5f159ac35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:19.723868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2e670-403d-4d16-86fb-c5a5f159ac35 service cinder] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:19.734918 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1396/2780] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:23.452345 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c856f8f-b13b-49b1-9150-29fa80a0d1e4 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:23.529913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c856f8f-b13b-49b1-9150-29fa80a0d1e4 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:23.535193 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:23.536089 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:23.536374 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:23.536664 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] PATH_INFO: `/v3/users/48d16bf316434f829358ffaafd217328` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:23.537921 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48d16bf316434f829358ffaafd217328)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:23.539891 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:23.716201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:23.725869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48d16bf316434f829358ffaafd217328'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:23.726978 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48d16bf316434f829358ffaafd217328'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:23.727655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48d16bf316434f829358ffaafd217328'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:23.773545 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48d16bf316434f829358ffaafd217328'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:23.790485 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48d16bf316434f829358ffaafd217328'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:23.804230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1385/2781] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:28:23 2022] DELETE /identity/v3/users/48d16bf316434f829358ffaafd217328 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:23.809148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82cc5e-ef22-4e78-ad1a-2750159a218c None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:23.813418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82cc5e-ef22-4e78-ad1a-2750159a218c None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:23.813418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82cc5e-ef22-4e78-ad1a-2750159a218c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:23.829969 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-8f82cc5e-ef22-4e78-ad1a-2750159a218c None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:23.831234 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8f82cc5e-ef22-4e78-ad1a-2750159a218c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:23.896401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1397/2782] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:28:23.906865 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:23.908537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:23.977511 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b25f3-c7dd-475d-8f8e-942998efb899 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:23.981875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25009411-fee3-4815-9224-3ed43901eeb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:23.982746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009411-fee3-4815-9224-3ed43901eeb7 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:23.983027 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009411-fee3-4815-9224-3ed43901eeb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:23.983276 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009411-fee3-4815-9224-3ed43901eeb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:23.993754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25009411-fee3-4815-9224-3ed43901eeb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:23.996193 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25009411-fee3-4815-9224-3ed43901eeb7 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:24.006890 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1386/2783] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:24.468162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82cc5e-ef22-4e78-ad1a-2750159a218c None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:24.471105 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8f82cc5e-ef22-4e78-ad1a-2750159a218c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:24.493323 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82cc5e-ef22-4e78-ad1a-2750159a218c None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:24.499138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:24.500176 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:24.500440 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:24.500676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] PATH_INFO: `/v3/projects/e74083c33c984f33bf55caa684b3fab7` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:24.501921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e74083c33c984f33bf55caa684b3fab7)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:24.503855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:24.656092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e74083c33c984f33bf55caa684b3fab7', 'request_id': 'req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:24.663481 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e74083c33c984f33bf55caa684b3fab7', 'request_id': 'req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:24.670501 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e74083c33c984f33bf55caa684b3fab7', 'request_id': 'req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:24.785413 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] The token cache is being invalidate because project e74083c33c984f33bf55caa684b3fab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:28:24.785638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:24.786295 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:24.787885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1398/2784] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:28:24 2022] DELETE /identity/v3/projects/e74083c33c984f33bf55caa684b3fab7 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:28:24.793267 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25009411-fee3-4815-9224-3ed43901eeb7 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:24.804282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25009411-fee3-4815-9224-3ed43901eeb7 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:24.879985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de454d0f-566d-4071-8e22-88ce70fd1ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:24.880878 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de454d0f-566d-4071-8e22-88ce70fd1ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:24.881162 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de454d0f-566d-4071-8e22-88ce70fd1ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:24.881440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de454d0f-566d-4071-8e22-88ce70fd1ecf admin admin] PATH_INFO: `/v3/users/ffc6373250d6461393155dc28a29af89` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:24.882689 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de454d0f-566d-4071-8e22-88ce70fd1ecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffc6373250d6461393155dc28a29af89)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:24.884633 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de454d0f-566d-4071-8e22-88ce70fd1ecf admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:25.091694 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-de454d0f-566d-4071-8e22-88ce70fd1ecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffc6373250d6461393155dc28a29af89'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:25.092282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-de454d0f-566d-4071-8e22-88ce70fd1ecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffc6373250d6461393155dc28a29af89'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:25.092906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-de454d0f-566d-4071-8e22-88ce70fd1ecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffc6373250d6461393155dc28a29af89'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:25.173431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-de454d0f-566d-4071-8e22-88ce70fd1ecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffc6373250d6461393155dc28a29af89'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:25.176747 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:25.191487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-de454d0f-566d-4071-8e22-88ce70fd1ecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffc6373250d6461393155dc28a29af89'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:25.209997 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1387/2785] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:28:24 2022] DELETE /identity/v3/users/ffc6373250d6461393155dc28a29af89 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:25.735588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:25.737121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:25.752875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e36bda-8d18-4bdc-9004-69dc2f6f65a6 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:25.783904 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:25.826849 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39653bc7-47b6-4f07-811e-b00bc0fce148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:25.828729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39653bc7-47b6-4f07-811e-b00bc0fce148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:25.828729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39653bc7-47b6-4f07-811e-b00bc0fce148 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:25.828729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39653bc7-47b6-4f07-811e-b00bc0fce148 admin admin] PATH_INFO: `/v3/projects/8cdfc81dd6024ea99d8d7a889675eea3` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:25.829614 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39653bc7-47b6-4f07-811e-b00bc0fce148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cdfc81dd6024ea99d8d7a889675eea3)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:25.831599 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39653bc7-47b6-4f07-811e-b00bc0fce148 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:26.042794 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-39653bc7-47b6-4f07-811e-b00bc0fce148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cdfc81dd6024ea99d8d7a889675eea3', 'request_id': 'req-39653bc7-47b6-4f07-811e-b00bc0fce148'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:26.050719 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-39653bc7-47b6-4f07-811e-b00bc0fce148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cdfc81dd6024ea99d8d7a889675eea3', 'request_id': 'req-39653bc7-47b6-4f07-811e-b00bc0fce148'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:26.056629 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-39653bc7-47b6-4f07-811e-b00bc0fce148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cdfc81dd6024ea99d8d7a889675eea3', 'request_id': 'req-39653bc7-47b6-4f07-811e-b00bc0fce148'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:26.191240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-39653bc7-47b6-4f07-811e-b00bc0fce148 admin admin] The token cache is being invalidate because project 8cdfc81dd6024ea99d8d7a889675eea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:28:26.191525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-39653bc7-47b6-4f07-811e-b00bc0fce148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:26.192205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-39653bc7-47b6-4f07-811e-b00bc0fce148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:26.193934 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1399/2786] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:28:25 2022] DELETE /identity/v3/projects/8cdfc81dd6024ea99d8d7a889675eea3 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:26.204887 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ecc49-1226-4dcb-a036-62ecbbbd0b3a None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:26.205534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ecc49-1226-4dcb-a036-62ecbbbd0b3a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:26.205534 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ecc49-1226-4dcb-a036-62ecbbbd0b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:26.227071 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-0c3ecc49-1226-4dcb-a036-62ecbbbd0b3a None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:26.228464 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-0c3ecc49-1226-4dcb-a036-62ecbbbd0b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:26.311049 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1388/2787] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:26.319788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfc6f2-6883-43a2-b897-1ce0ae644dd1 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:26.320286 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfc6f2-6883-43a2-b897-1ce0ae644dd1 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:26.320749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfc6f2-6883-43a2-b897-1ce0ae644dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:26.346483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-99cfc6f2-6883-43a2-b897-1ce0ae644dd1 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:26.347743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-99cfc6f2-6883-43a2-b897-1ce0ae644dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:26.357150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1400/2788] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:26.361903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ecc49-1226-4dcb-a036-62ecbbbd0b3a None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:26.373052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ecc49-1226-4dcb-a036-62ecbbbd0b3a None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:26.378601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106a159-50cb-4320-994e-a1b3853fab6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:26.379300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106a159-50cb-4320-994e-a1b3853fab6e admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:26.379589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106a159-50cb-4320-994e-a1b3853fab6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:26.379868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106a159-50cb-4320-994e-a1b3853fab6e admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:26.380622 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106a159-50cb-4320-994e-a1b3853fab6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:28:26.380966 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106a159-50cb-4320-994e-a1b3853fab6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:26.382715 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106a159-50cb-4320-994e-a1b3853fab6e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:26.396618 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1389/2789] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:28:26.402601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfc6f2-6883-43a2-b897-1ce0ae644dd1 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:26.413282 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfc6f2-6883-43a2-b897-1ce0ae644dd1 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:26.419022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef69b5-7e86-465a-b457-a15023891cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:26.419750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef69b5-7e86-465a-b457-a15023891cff admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:26.420009 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef69b5-7e86-465a-b457-a15023891cff admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:26.420287 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef69b5-7e86-465a-b457-a15023891cff admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:26.420995 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ef69b5-7e86-465a-b457-a15023891cff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:26.422955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ef69b5-7e86-465a-b457-a15023891cff admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:26.528690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1401/2790] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:28:26 2022] POST /identity/v3/projects => generated 397 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:28:26.534039 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106a159-50cb-4320-994e-a1b3853fab6e admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:26.545420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106a159-50cb-4320-994e-a1b3853fab6e admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:26.616318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca4a13-5189-46c7-8266-8a9b5f4d2b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:26.616906 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca4a13-5189-46c7-8266-8a9b5f4d2b3a admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:26.617192 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca4a13-5189-46c7-8266-8a9b5f4d2b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:26.617473 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca4a13-5189-46c7-8266-8a9b5f4d2b3a admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:26.618163 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca4a13-5189-46c7-8266-8a9b5f4d2b3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:26.620334 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca4a13-5189-46c7-8266-8a9b5f4d2b3a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:26.742857 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1390/2791] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:28:26 2022] POST /identity/v3/users => generated 299 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:28:26.747770 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef69b5-7e86-465a-b457-a15023891cff admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:26.763183 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef69b5-7e86-465a-b457-a15023891cff admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:26.768193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02990ac9-13b5-48cf-bfae-596f259b1c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:26.768873 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02990ac9-13b5-48cf-bfae-596f259b1c9c admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:26.769183 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02990ac9-13b5-48cf-bfae-596f259b1c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:26.769463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02990ac9-13b5-48cf-bfae-596f259b1c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:26.770190 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02990ac9-13b5-48cf-bfae-596f259b1c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:26.771872 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02990ac9-13b5-48cf-bfae-596f259b1c9c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:26.784241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1402/2792] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:28:26.788238 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca4a13-5189-46c7-8266-8a9b5f4d2b3a admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:26.800024 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca4a13-5189-46c7-8266-8a9b5f4d2b3a admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:26.804864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f90d43-46d2-4f43-a06a-d49140b8be74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:26.805542 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f90d43-46d2-4f43-a06a-d49140b8be74 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:26.805859 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f90d43-46d2-4f43-a06a-d49140b8be74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:26.806095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f90d43-46d2-4f43-a06a-d49140b8be74 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:26.807040 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f90d43-46d2-4f43-a06a-d49140b8be74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:26.808757 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f90d43-46d2-4f43-a06a-d49140b8be74 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:26.821586 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1391/2793] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:28:26.825925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02990ac9-13b5-48cf-bfae-596f259b1c9c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:26.836632 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02990ac9-13b5-48cf-bfae-596f259b1c9c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:26.842369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef10f8-a1b3-42a8-9e52-588f23e464ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:26.843297 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef10f8-a1b3-42a8-9e52-588f23e464ba admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:26.843582 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef10f8-a1b3-42a8-9e52-588f23e464ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:26.843857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef10f8-a1b3-42a8-9e52-588f23e464ba admin admin] PATH_INFO: `/v3/projects/520ca45ae57e4f05b1c81d659393cd4f/users/b0d19d82819242cca81df0b4e4dcdd11/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:26.860562 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef10f8-a1b3-42a8-9e52-588f23e464ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=520ca45ae57e4f05b1c81d659393cd4f, user_id=b0d19d82819242cca81df0b4e4dcdd11, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:26.863255 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef10f8-a1b3-42a8-9e52-588f23e464ba admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:26.939203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1403/2794] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:28:26 2022] PUT /identity/v3/projects/520ca45ae57e4f05b1c81d659393cd4f/users/b0d19d82819242cca81df0b4e4dcdd11/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:26.950509 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bd50c-78b2-4034-90ce-7e74a0036f6a None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:26.954032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bd50c-78b2-4034-90ce-7e74a0036f6a None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:26.954032 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bd50c-78b2-4034-90ce-7e74a0036f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:26.984557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-d37bd50c-78b2-4034-90ce-7e74a0036f6a None None] MFA Rules not processed for user `b0d19d82819242cca81df0b4e4dcdd11`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:26.988321 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d37bd50c-78b2-4034-90ce-7e74a0036f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:27.019341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:27.060428 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1392/2795] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:27.065089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef10f8-a1b3-42a8-9e52-588f23e464ba admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:27.134075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:27.158494 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef10f8-a1b3-42a8-9e52-588f23e464ba admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:27.168181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532d0cf-ae03-4c04-b730-b68477b8bd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:27.173283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532d0cf-ae03-4c04-b730-b68477b8bd24 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:27.173283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532d0cf-ae03-4c04-b730-b68477b8bd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:27.173283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532d0cf-ae03-4c04-b730-b68477b8bd24 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:27.173283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532d0cf-ae03-4c04-b730-b68477b8bd24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:27.173283 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532d0cf-ae03-4c04-b730-b68477b8bd24 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:27.243721 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1404/2796] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:28:27 2022] POST /identity/v3/projects => generated 397 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:28:27.248652 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bd50c-78b2-4034-90ce-7e74a0036f6a None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:27.267739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bd50c-78b2-4034-90ce-7e74a0036f6a None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:27.344259 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe988b65-eba4-43b1-b54f-666f2208c086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:27.344961 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe988b65-eba4-43b1-b54f-666f2208c086 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:27.345251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe988b65-eba4-43b1-b54f-666f2208c086 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:27.345528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe988b65-eba4-43b1-b54f-666f2208c086 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:27.346234 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe988b65-eba4-43b1-b54f-666f2208c086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:27.348260 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe988b65-eba4-43b1-b54f-666f2208c086 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:27.409230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1393/2797] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:28:27 2022] POST /identity/v3/users => generated 305 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:28:27.413558 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532d0cf-ae03-4c04-b730-b68477b8bd24 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:27.424839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532d0cf-ae03-4c04-b730-b68477b8bd24 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:27.432257 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a0115-ce34-408a-b2ab-1ab5a9414f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:27.433073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a0115-ce34-408a-b2ab-1ab5a9414f33 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:27.433401 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a0115-ce34-408a-b2ab-1ab5a9414f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:27.433724 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a0115-ce34-408a-b2ab-1ab5a9414f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:27.434693 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a0115-ce34-408a-b2ab-1ab5a9414f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:27.436369 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a0115-ce34-408a-b2ab-1ab5a9414f33 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:27.447982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1405/2798] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:28:27.453512 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe988b65-eba4-43b1-b54f-666f2208c086 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:27.466836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe988b65-eba4-43b1-b54f-666f2208c086 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:27.471895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6af2a7-69b7-4717-99d9-277c611d15c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:27.472808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6af2a7-69b7-4717-99d9-277c611d15c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:27.473092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6af2a7-69b7-4717-99d9-277c611d15c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:27.473377 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6af2a7-69b7-4717-99d9-277c611d15c2 admin admin] PATH_INFO: `/v3/projects/dce780b8a6064fc3b03c27a827a3164e/users/60dbd62d464b4ae897c37e19ee6af909/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:27.490770 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6af2a7-69b7-4717-99d9-277c611d15c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce780b8a6064fc3b03c27a827a3164e, user_id=60dbd62d464b4ae897c37e19ee6af909, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:27.493481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6af2a7-69b7-4717-99d9-277c611d15c2 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:27.599437 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1394/2799] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:28:27 2022] PUT /identity/v3/projects/dce780b8a6064fc3b03c27a827a3164e/users/60dbd62d464b4ae897c37e19ee6af909/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:27.604866 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a0115-ce34-408a-b2ab-1ab5a9414f33 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:27.616326 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a0115-ce34-408a-b2ab-1ab5a9414f33 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:27.717291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85197ef-3abe-4126-85d7-b97bc3508bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:27.718027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85197ef-3abe-4126-85d7-b97bc3508bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:27.718313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85197ef-3abe-4126-85d7-b97bc3508bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:27.718625 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85197ef-3abe-4126-85d7-b97bc3508bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:27.719379 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85197ef-3abe-4126-85d7-b97bc3508bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:27.721054 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85197ef-3abe-4126-85d7-b97bc3508bb4 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:27.735876 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1406/2800] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:28:27.740196 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6af2a7-69b7-4717-99d9-277c611d15c2 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:27.761563 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6af2a7-69b7-4717-99d9-277c611d15c2 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:27.768167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc53d5-4954-4271-a5b4-9a2c78eff82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:27.771003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc53d5-4954-4271-a5b4-9a2c78eff82c admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:27.771300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc53d5-4954-4271-a5b4-9a2c78eff82c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:27.771594 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc53d5-4954-4271-a5b4-9a2c78eff82c admin admin] PATH_INFO: `/v3/projects/dce780b8a6064fc3b03c27a827a3164e/users/60dbd62d464b4ae897c37e19ee6af909/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:27.774478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc53d5-4954-4271-a5b4-9a2c78eff82c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce780b8a6064fc3b03c27a827a3164e, user_id=60dbd62d464b4ae897c37e19ee6af909, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:27.778703 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc53d5-4954-4271-a5b4-9a2c78eff82c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:27.797209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1395/2801] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:28:27 2022] PUT /identity/v3/projects/dce780b8a6064fc3b03c27a827a3164e/users/60dbd62d464b4ae897c37e19ee6af909/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:27.878789 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d63a12-eb8c-4788-83be-d93093e44416 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:27.879423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d63a12-eb8c-4788-83be-d93093e44416 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:27.879423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d63a12-eb8c-4788-83be-d93093e44416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:27.918521 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-53d63a12-eb8c-4788-83be-d93093e44416 None None] MFA Rules not processed for user `60dbd62d464b4ae897c37e19ee6af909`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:27.919837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-53d63a12-eb8c-4788-83be-d93093e44416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:28.011517 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1407/2802] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:28.022551 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc53d5-4954-4271-a5b4-9a2c78eff82c admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:28.024333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9efc53d5-4954-4271-a5b4-9a2c78eff82c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:28.098207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc53d5-4954-4271-a5b4-9a2c78eff82c admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:28.102203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbb41c-210a-4e93-b7c2-2f96bf6837f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:28.102933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbb41c-210a-4e93-b7c2-2f96bf6837f8 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:28.103228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbb41c-210a-4e93-b7c2-2f96bf6837f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:28.103514 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbb41c-210a-4e93-b7c2-2f96bf6837f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:28.116545 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbb41c-210a-4e93-b7c2-2f96bf6837f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:28.118310 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbb41c-210a-4e93-b7c2-2f96bf6837f8 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:28.131848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1396/2803] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:28.150701 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d63a12-eb8c-4788-83be-d93093e44416 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:28.152459 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-53d63a12-eb8c-4788-83be-d93093e44416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:28.210290 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:28.238437 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d63a12-eb8c-4788-83be-d93093e44416 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:28.248506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c631889-14af-4f8d-b0f0-62378822f34b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:28.249203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c631889-14af-4f8d-b0f0-62378822f34b service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:28.249597 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c631889-14af-4f8d-b0f0-62378822f34b service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:28.250269 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c631889-14af-4f8d-b0f0-62378822f34b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:28.253188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-4c631889-14af-4f8d-b0f0-62378822f34b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:28.268221 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c631889-14af-4f8d-b0f0-62378822f34b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:28.270303 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c631889-14af-4f8d-b0f0-62378822f34b service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:28.282835 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1408/2804] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:28.412150 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbb41c-210a-4e93-b7c2-2f96bf6837f8 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:28.422095 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbb41c-210a-4e93-b7c2-2f96bf6837f8 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:28.426054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1fe2d-1d35-48d4-9952-7bee86538b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:28.430573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1fe2d-1d35-48d4-9952-7bee86538b29 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:28.430573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1fe2d-1d35-48d4-9952-7bee86538b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:28.430573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1fe2d-1d35-48d4-9952-7bee86538b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:28.459237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:28.515926 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1fe2d-1d35-48d4-9952-7bee86538b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:28.517703 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1fe2d-1d35-48d4-9952-7bee86538b29 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:28.530190 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1397/2805] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:34.710372 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c631889-14af-4f8d-b0f0-62378822f34b service placement] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:34.794158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c631889-14af-4f8d-b0f0-62378822f34b service placement] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:34.800393 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:34.801379 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:34.801675 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:34.801958 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] PATH_INFO: `/v3/users/b0d19d82819242cca81df0b4e4dcdd11` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:34.803346 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0d19d82819242cca81df0b4e4dcdd11)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:34.805313 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:34.997986 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0d19d82819242cca81df0b4e4dcdd11'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:34.998715 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0d19d82819242cca81df0b4e4dcdd11'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:35.007370 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0d19d82819242cca81df0b4e4dcdd11'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:35.023186 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0d19d82819242cca81df0b4e4dcdd11'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:35.081460 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0d19d82819242cca81df0b4e4dcdd11'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:35.089374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1409/2806] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:28:34 2022] DELETE /identity/v3/users/b0d19d82819242cca81df0b4e4dcdd11 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:35.094545 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf2d70-a03a-4b31-8802-6b2c2ac5877b None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:35.095127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf2d70-a03a-4b31-8802-6b2c2ac5877b None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:35.095127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf2d70-a03a-4b31-8802-6b2c2ac5877b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:35.116645 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-daaf2d70-a03a-4b31-8802-6b2c2ac5877b None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:35.117885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-daaf2d70-a03a-4b31-8802-6b2c2ac5877b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:35.205519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1398/2807] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:35.219973 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:35.221690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:35.299241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12710eba-523a-46da-a3b2-c6231141b56d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:35.303857 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55307fc8-d5be-4997-8bd1-d3f71261e243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:35.304699 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55307fc8-d5be-4997-8bd1-d3f71261e243 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:35.305057 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55307fc8-d5be-4997-8bd1-d3f71261e243 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:35.305455 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55307fc8-d5be-4997-8bd1-d3f71261e243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:35.314994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:35.318821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55307fc8-d5be-4997-8bd1-d3f71261e243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:35.320642 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55307fc8-d5be-4997-8bd1-d3f71261e243 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:35.334052 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1410/2808] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:35.809640 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf2d70-a03a-4b31-8802-6b2c2ac5877b None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:35.811415 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-daaf2d70-a03a-4b31-8802-6b2c2ac5877b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:35.827589 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf2d70-a03a-4b31-8802-6b2c2ac5877b None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:35.832653 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:35.833598 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:35.834554 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:35.834842 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] PATH_INFO: `/v3/projects/520ca45ae57e4f05b1c81d659393cd4f` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:35.836369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=520ca45ae57e4f05b1c81d659393cd4f)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:35.839166 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:35.984111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '520ca45ae57e4f05b1c81d659393cd4f', 'request_id': 'req-18290005-cda5-470c-a4aa-1acd32bb6258'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:36.053253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '520ca45ae57e4f05b1c81d659393cd4f', 'request_id': 'req-18290005-cda5-470c-a4aa-1acd32bb6258'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:36.063650 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '520ca45ae57e4f05b1c81d659393cd4f', 'request_id': 'req-18290005-cda5-470c-a4aa-1acd32bb6258'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:36.077409 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] The token cache is being invalidate because project 520ca45ae57e4f05b1c81d659393cd4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:28:36.077708 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:36.078492 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:36.080364 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1399/2809] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:28:35 2022] DELETE /identity/v3/projects/520ca45ae57e4f05b1c81d659393cd4f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:28:36.084661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55307fc8-d5be-4997-8bd1-d3f71261e243 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:36.102827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55307fc8-d5be-4997-8bd1-d3f71261e243 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:36.169553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6714058-4e1f-4bc9-8be8-5c22ed06c90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:36.170483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6714058-4e1f-4bc9-8be8-5c22ed06c90a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:36.170769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6714058-4e1f-4bc9-8be8-5c22ed06c90a admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:36.171048 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6714058-4e1f-4bc9-8be8-5c22ed06c90a admin admin] PATH_INFO: `/v3/users/60dbd62d464b4ae897c37e19ee6af909` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:36.172307 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6714058-4e1f-4bc9-8be8-5c22ed06c90a admin admin] RBAC: Authorizing `identity:delete_user(user_id=60dbd62d464b4ae897c37e19ee6af909)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:36.174253 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6714058-4e1f-4bc9-8be8-5c22ed06c90a admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:36.366588 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d6714058-4e1f-4bc9-8be8-5c22ed06c90a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60dbd62d464b4ae897c37e19ee6af909'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:36.367265 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d6714058-4e1f-4bc9-8be8-5c22ed06c90a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60dbd62d464b4ae897c37e19ee6af909'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:36.378512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d6714058-4e1f-4bc9-8be8-5c22ed06c90a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60dbd62d464b4ae897c37e19ee6af909'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:36.394661 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d6714058-4e1f-4bc9-8be8-5c22ed06c90a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60dbd62d464b4ae897c37e19ee6af909'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:36.464182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-d6714058-4e1f-4bc9-8be8-5c22ed06c90a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60dbd62d464b4ae897c37e19ee6af909'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:36.470931 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1411/2810] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:28:36 2022] DELETE /identity/v3/users/60dbd62d464b4ae897c37e19ee6af909 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:36.981214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:36.982861 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:36.998287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18290005-cda5-470c-a4aa-1acd32bb6258 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:37.056487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:37.073076 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:37.074560 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:37.074849 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:37.075126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74 admin admin] PATH_INFO: `/v3/projects/dce780b8a6064fc3b03c27a827a3164e` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:37.076676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce780b8a6064fc3b03c27a827a3164e)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:37.078767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:37.194075 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce780b8a6064fc3b03c27a827a3164e', 'request_id': 'req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:37.275223 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce780b8a6064fc3b03c27a827a3164e', 'request_id': 'req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:37.286248 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce780b8a6064fc3b03c27a827a3164e', 'request_id': 'req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:37.306189 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74 admin admin] The token cache is being invalidate because project dce780b8a6064fc3b03c27a827a3164e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:28:37.306972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:37.308284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-9acb39f6-dc3b-4b6e-aeca-f5e62be0cf74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:37.310296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1400/2811] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:28:36 2022] DELETE /identity/v3/projects/dce780b8a6064fc3b03c27a827a3164e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:37.317821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f430a-7b06-4535-b676-d78e4faf169f None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:37.318743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f430a-7b06-4535-b676-d78e4faf169f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:37.318743 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f430a-7b06-4535-b676-d78e4faf169f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:37.343946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ca6f430a-7b06-4535-b676-d78e4faf169f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:37.345318 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ca6f430a-7b06-4535-b676-d78e4faf169f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:37.427071 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1412/2812] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:37.436435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935cbae-9187-4fdd-b102-4565ef9fc717 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:37.436707 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935cbae-9187-4fdd-b102-4565ef9fc717 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:37.436992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935cbae-9187-4fdd-b102-4565ef9fc717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:37.461226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3935cbae-9187-4fdd-b102-4565ef9fc717 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:37.462530 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3935cbae-9187-4fdd-b102-4565ef9fc717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:37.471129 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1401/2813] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:28:37.475951 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f430a-7b06-4535-b676-d78e4faf169f None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:37.488725 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f430a-7b06-4535-b676-d78e4faf169f None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:37.493796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770be38-7680-4ade-aae1-52ee7e1de95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:37.495416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770be38-7680-4ade-aae1-52ee7e1de95c admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:37.495744 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770be38-7680-4ade-aae1-52ee7e1de95c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:37.496019 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770be38-7680-4ade-aae1-52ee7e1de95c admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:37.496785 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770be38-7680-4ade-aae1-52ee7e1de95c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:28:37.497108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770be38-7680-4ade-aae1-52ee7e1de95c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:37.498809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770be38-7680-4ade-aae1-52ee7e1de95c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:37.513823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1413/2814] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:28:37.520050 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935cbae-9187-4fdd-b102-4565ef9fc717 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:37.536172 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935cbae-9187-4fdd-b102-4565ef9fc717 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:37.542160 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd900c-a14a-435c-b5da-7afdc6a3d282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:37.543010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd900c-a14a-435c-b5da-7afdc6a3d282 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:37.543341 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd900c-a14a-435c-b5da-7afdc6a3d282 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:37.543661 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd900c-a14a-435c-b5da-7afdc6a3d282 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:37.544506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd900c-a14a-435c-b5da-7afdc6a3d282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:37.546755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd900c-a14a-435c-b5da-7afdc6a3d282 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:37.655302 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1402/2815] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:28:37 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:28:37.659744 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770be38-7680-4ade-aae1-52ee7e1de95c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:37.665357 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:37.671823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770be38-7680-4ade-aae1-52ee7e1de95c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:37.742618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82ba89-0420-4f81-81dc-2a8e4546dc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:37.743320 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82ba89-0420-4f81-81dc-2a8e4546dc89 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:37.743618 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82ba89-0420-4f81-81dc-2a8e4546dc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:37.743896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82ba89-0420-4f81-81dc-2a8e4546dc89 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:37.744758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82ba89-0420-4f81-81dc-2a8e4546dc89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:37.746946 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82ba89-0420-4f81-81dc-2a8e4546dc89 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:37.868123 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1414/2816] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:28:37 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:28:37.873623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd900c-a14a-435c-b5da-7afdc6a3d282 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:37.885576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd900c-a14a-435c-b5da-7afdc6a3d282 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:37.890928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72587b7-3113-416e-aeee-9026517ac481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:37.891626 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72587b7-3113-416e-aeee-9026517ac481 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:37.891912 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72587b7-3113-416e-aeee-9026517ac481 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:37.892188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72587b7-3113-416e-aeee-9026517ac481 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:37.892940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72587b7-3113-416e-aeee-9026517ac481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:37.894597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72587b7-3113-416e-aeee-9026517ac481 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:37.906883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1403/2817] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:28:37.912566 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82ba89-0420-4f81-81dc-2a8e4546dc89 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:37.927676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82ba89-0420-4f81-81dc-2a8e4546dc89 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:37.934409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773d19d-19de-43be-9326-0226fd40a3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:37.935108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773d19d-19de-43be-9326-0226fd40a3bb admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:37.935398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773d19d-19de-43be-9326-0226fd40a3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:37.935682 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773d19d-19de-43be-9326-0226fd40a3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:37.936422 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773d19d-19de-43be-9326-0226fd40a3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:37.938072 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773d19d-19de-43be-9326-0226fd40a3bb admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:37.949489 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1415/2818] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:28:37.953860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72587b7-3113-416e-aeee-9026517ac481 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:37.968373 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72587b7-3113-416e-aeee-9026517ac481 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:37.973273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0995273-a5d2-4229-8e3e-93731e0f1279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:37.974184 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0995273-a5d2-4229-8e3e-93731e0f1279 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:37.974505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0995273-a5d2-4229-8e3e-93731e0f1279 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:37.974788 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0995273-a5d2-4229-8e3e-93731e0f1279 admin admin] PATH_INFO: `/v3/projects/b7ad51803f824656937a069ec6bc21f8/users/f46489a3c9f246c084850896d8637bd4/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:37.991601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0995273-a5d2-4229-8e3e-93731e0f1279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ad51803f824656937a069ec6bc21f8, user_id=f46489a3c9f246c084850896d8637bd4, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:37.994600 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0995273-a5d2-4229-8e3e-93731e0f1279 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:38.070539 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1404/2819] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:28:37 2022] PUT /identity/v3/projects/b7ad51803f824656937a069ec6bc21f8/users/f46489a3c9f246c084850896d8637bd4/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:38.088662 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651e175-e075-4b95-be52-b1d09ba14a4a None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:38.089182 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651e175-e075-4b95-be52-b1d09ba14a4a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:38.089530 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651e175-e075-4b95-be52-b1d09ba14a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:38.123022 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-8651e175-e075-4b95-be52-b1d09ba14a4a None None] MFA Rules not processed for user `f46489a3c9f246c084850896d8637bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:38.124186 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-8651e175-e075-4b95-be52-b1d09ba14a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:38.200715 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1416/2820] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:38.213366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0995273-a5d2-4229-8e3e-93731e0f1279 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:38.215574 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a0995273-a5d2-4229-8e3e-93731e0f1279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:38.289199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0995273-a5d2-4229-8e3e-93731e0f1279 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:38.293369 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e2617-0dc0-4877-a556-a93ea2b8ea5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:38.294033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e2617-0dc0-4877-a556-a93ea2b8ea5d service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:38.294318 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e2617-0dc0-4877-a556-a93ea2b8ea5d service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:38.294624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e2617-0dc0-4877-a556-a93ea2b8ea5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:38.303944 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e2617-0dc0-4877-a556-a93ea2b8ea5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:38.311954 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e2617-0dc0-4877-a556-a93ea2b8ea5d service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:38.319323 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1405/2821] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:28:38.339901 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651e175-e075-4b95-be52-b1d09ba14a4a None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:38.420802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651e175-e075-4b95-be52-b1d09ba14a4a None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:38.425471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:38.426589 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:38.426882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:38.427168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] PATH_INFO: `/v3/users/f46489a3c9f246c084850896d8637bd4` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:38.428528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f46489a3c9f246c084850896d8637bd4)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:38.430528 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:38.444461 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:38.581169 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f46489a3c9f246c084850896d8637bd4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:38.581884 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f46489a3c9f246c084850896d8637bd4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:38.590007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f46489a3c9f246c084850896d8637bd4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:38.602602 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f46489a3c9f246c084850896d8637bd4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:38.680572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f46489a3c9f246c084850896d8637bd4'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:38.687319 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1417/2822] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:28:38 2022] DELETE /identity/v3/users/f46489a3c9f246c084850896d8637bd4 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:38.692087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e606de2-b592-404a-9bac-a4374299b02f None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:38.698350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e606de2-b592-404a-9bac-a4374299b02f None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:38.698350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e606de2-b592-404a-9bac-a4374299b02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:38.714344 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-8e606de2-b592-404a-9bac-a4374299b02f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:38.715828 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8e606de2-b592-404a-9bac-a4374299b02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:38.725895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:38.792014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1406/2823] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:38.802060 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:38.803565 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:38.863595 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832cf72-295b-4f4a-ac99-78c9f79ea6e8 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:38.870821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3475813-be4d-408e-94b5-0f7c96ec803b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:38.871490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3475813-be4d-408e-94b5-0f7c96ec803b service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:38.871830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3475813-be4d-408e-94b5-0f7c96ec803b service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:38.872083 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3475813-be4d-408e-94b5-0f7c96ec803b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:38.881837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3475813-be4d-408e-94b5-0f7c96ec803b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:38.883321 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3475813-be4d-408e-94b5-0f7c96ec803b service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:38.894397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1418/2824] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:39.318296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e606de2-b592-404a-9bac-a4374299b02f None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:39.319820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8e606de2-b592-404a-9bac-a4374299b02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:39.340445 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e606de2-b592-404a-9bac-a4374299b02f None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:39.347134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:39.348578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:39.349036 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:39.349420 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a admin admin] PATH_INFO: `/v3/projects/b7ad51803f824656937a069ec6bc21f8` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:39.351214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ad51803f824656937a069ec6bc21f8)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:39.354126 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:39.488842 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ad51803f824656937a069ec6bc21f8', 'request_id': 'req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:39.549079 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ad51803f824656937a069ec6bc21f8', 'request_id': 'req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:39.555293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ad51803f824656937a069ec6bc21f8', 'request_id': 'req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:39.568081 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a admin admin] The token cache is being invalidate because project b7ad51803f824656937a069ec6bc21f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:28:39.568294 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:39.569092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-a2f0490a-56cc-4733-a57d-7f3d5f325b1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:28:39.570875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1407/2825] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:28:39 2022] DELETE /identity/v3/projects/b7ad51803f824656937a069ec6bc21f8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:39.576590 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4eb56-43e5-4cb5-97ad-262b9553326e None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:39.577276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4eb56-43e5-4cb5-97ad-262b9553326e None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:39.577276 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4eb56-43e5-4cb5-97ad-262b9553326e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:39.598201 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-d9d4eb56-43e5-4cb5-97ad-262b9553326e None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:39.599490 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-d9d4eb56-43e5-4cb5-97ad-262b9553326e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:39.662050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1419/2826] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:39.670293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108bcaf-93cb-4ef1-b9f3-3bb0effcf428 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:39.670635 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108bcaf-93cb-4ef1-b9f3-3bb0effcf428 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:39.670933 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108bcaf-93cb-4ef1-b9f3-3bb0effcf428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:39.692298 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-3108bcaf-93cb-4ef1-b9f3-3bb0effcf428 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:39.693469 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-3108bcaf-93cb-4ef1-b9f3-3bb0effcf428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:39.702737 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1408/2827] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:39.707224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4eb56-43e5-4cb5-97ad-262b9553326e None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:39.717254 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4eb56-43e5-4cb5-97ad-262b9553326e None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:39.721828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9f687-073d-4869-890e-45ce170b5f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:39.722522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9f687-073d-4869-890e-45ce170b5f05 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:39.722811 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9f687-073d-4869-890e-45ce170b5f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:39.723089 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9f687-073d-4869-890e-45ce170b5f05 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:39.723808 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9f687-073d-4869-890e-45ce170b5f05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:28:39.724117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9f687-073d-4869-890e-45ce170b5f05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:39.725709 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9f687-073d-4869-890e-45ce170b5f05 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:39.738396 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1420/2828] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:28:39.743153 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108bcaf-93cb-4ef1-b9f3-3bb0effcf428 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:39.753570 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108bcaf-93cb-4ef1-b9f3-3bb0effcf428 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:39.758586 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60b416-15f7-4f93-bf6b-9f35fa5e6eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:39.759308 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60b416-15f7-4f93-bf6b-9f35fa5e6eff admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:39.759713 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60b416-15f7-4f93-bf6b-9f35fa5e6eff admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:39.760738 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60b416-15f7-4f93-bf6b-9f35fa5e6eff admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:39.761411 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60b416-15f7-4f93-bf6b-9f35fa5e6eff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:39.763368 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60b416-15f7-4f93-bf6b-9f35fa5e6eff admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:39.848598 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1409/2829] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:28:39 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:28:39.852736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9f687-073d-4869-890e-45ce170b5f05 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:39.864227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9f687-073d-4869-890e-45ce170b5f05 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:39.890023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:39.935943 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4dd4d-53ec-4266-8f45-42875513e443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:39.936670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4dd4d-53ec-4266-8f45-42875513e443 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:39.936963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4dd4d-53ec-4266-8f45-42875513e443 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:39.937303 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4dd4d-53ec-4266-8f45-42875513e443 admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:39.937954 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4dd4d-53ec-4266-8f45-42875513e443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:39.939925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4dd4d-53ec-4266-8f45-42875513e443 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:40.012879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1421/2830] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:28:39 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:28:40.017334 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60b416-15f7-4f93-bf6b-9f35fa5e6eff admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:40.028312 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60b416-15f7-4f93-bf6b-9f35fa5e6eff admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:40.033805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc1424-0f8d-469d-8b15-62b79bb68547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:40.033805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc1424-0f8d-469d-8b15-62b79bb68547 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:40.034211 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc1424-0f8d-469d-8b15-62b79bb68547 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:40.034407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc1424-0f8d-469d-8b15-62b79bb68547 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:40.035054 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc1424-0f8d-469d-8b15-62b79bb68547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:40.036670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc1424-0f8d-469d-8b15-62b79bb68547 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:40.048378 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1410/2831] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:28:40.052571 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4dd4d-53ec-4266-8f45-42875513e443 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:40.063965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4dd4d-53ec-4266-8f45-42875513e443 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:40.069257 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac120b11-a806-4e15-9364-2bc22d1fd5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:40.069945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac120b11-a806-4e15-9364-2bc22d1fd5ca admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:40.070241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac120b11-a806-4e15-9364-2bc22d1fd5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:40.070564 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac120b11-a806-4e15-9364-2bc22d1fd5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:40.071302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac120b11-a806-4e15-9364-2bc22d1fd5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:40.073077 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac120b11-a806-4e15-9364-2bc22d1fd5ca admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:40.084820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1422/2832] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:28:40.089019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc1424-0f8d-469d-8b15-62b79bb68547 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:40.099273 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc1424-0f8d-469d-8b15-62b79bb68547 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:40.104433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a0610-2fd9-4110-a89b-92fb488af5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:40.105573 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a0610-2fd9-4110-a89b-92fb488af5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:40.105855 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a0610-2fd9-4110-a89b-92fb488af5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:40.106130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a0610-2fd9-4110-a89b-92fb488af5cd admin admin] PATH_INFO: `/v3/projects/d21026bdfd1d47c2a30e97fce3fb70ae/users/160f86cbd46d49daaa3fe5edcb82e64a/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:40.122021 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a0610-2fd9-4110-a89b-92fb488af5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d21026bdfd1d47c2a30e97fce3fb70ae, user_id=160f86cbd46d49daaa3fe5edcb82e64a, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:40.124743 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a0610-2fd9-4110-a89b-92fb488af5cd admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:40.202171 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1411/2833] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:28:40 2022] PUT /identity/v3/projects/d21026bdfd1d47c2a30e97fce3fb70ae/users/160f86cbd46d49daaa3fe5edcb82e64a/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:40.212906 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa7710-587b-40a4-a433-ea4609dc9e40 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:40.213202 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa7710-587b-40a4-a433-ea4609dc9e40 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:40.213556 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa7710-587b-40a4-a433-ea4609dc9e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:40.242846 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-15fa7710-587b-40a4-a433-ea4609dc9e40 None None] MFA Rules not processed for user `160f86cbd46d49daaa3fe5edcb82e64a`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:40.244792 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-15fa7710-587b-40a4-a433-ea4609dc9e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:40.315245 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1423/2834] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:28:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:40.327819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a0610-2fd9-4110-a89b-92fb488af5cd admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:40.329567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d32a0610-2fd9-4110-a89b-92fb488af5cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:40.372860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:40.391237 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a0610-2fd9-4110-a89b-92fb488af5cd admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:40.395111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607e8f4-b781-4632-8138-0d1323598922 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28954d67689a46ee975fdfc9139e41eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:40.395811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607e8f4-b781-4632-8138-0d1323598922 service glance] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:40.396103 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607e8f4-b781-4632-8138-0d1323598922 service glance] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:40.396414 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607e8f4-b781-4632-8138-0d1323598922 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:40.407059 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607e8f4-b781-4632-8138-0d1323598922 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:40.408529 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607e8f4-b781-4632-8138-0d1323598922 service glance] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:40.419996 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1412/2835] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:50.312226 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:50.463532 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=10 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:06.262523 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa7710-587b-40a4-a433-ea4609dc9e40 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:06.327110 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa7710-587b-40a4-a433-ea4609dc9e40 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:06.331961 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:06.332860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:06.333161 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:06.333443 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] PATH_INFO: `/v3/users/160f86cbd46d49daaa3fe5edcb82e64a` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:06.334696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] RBAC: Authorizing `identity:delete_user(user_id=160f86cbd46d49daaa3fe5edcb82e64a)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:06.336642 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:06.488802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '160f86cbd46d49daaa3fe5edcb82e64a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:06.489466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '160f86cbd46d49daaa3fe5edcb82e64a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:06.496485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '160f86cbd46d49daaa3fe5edcb82e64a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:06.508755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '160f86cbd46d49daaa3fe5edcb82e64a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:06.554412 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '160f86cbd46d49daaa3fe5edcb82e64a'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:06.561560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1424/2836] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:29:06 2022] DELETE /identity/v3/users/160f86cbd46d49daaa3fe5edcb82e64a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:29:06.566518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d435e71-f52d-4a3d-921a-8c5824bbcd02 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:06.567287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d435e71-f52d-4a3d-921a-8c5824bbcd02 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:06.567287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d435e71-f52d-4a3d-921a-8c5824bbcd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:06.587807 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-6d435e71-f52d-4a3d-921a-8c5824bbcd02 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:06.589321 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6d435e71-f52d-4a3d-921a-8c5824bbcd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:06.649289 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1413/2837] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:29:06.659458 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:06.660963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:06.724122 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6e5c9-0208-42c0-8045-74b14b3d204d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:06.727938 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46512d7b-f4d3-49e7-b322-8ec8448f5cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:06.728677 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46512d7b-f4d3-49e7-b322-8ec8448f5cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:06.729907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46512d7b-f4d3-49e7-b322-8ec8448f5cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:06.730238 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46512d7b-f4d3-49e7-b322-8ec8448f5cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:06.740668 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46512d7b-f4d3-49e7-b322-8ec8448f5cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:06.742113 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46512d7b-f4d3-49e7-b322-8ec8448f5cc6 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:06.753823 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1425/2838] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:29:07.305127 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d435e71-f52d-4a3d-921a-8c5824bbcd02 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:07.314875 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-6d435e71-f52d-4a3d-921a-8c5824bbcd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:07.320735 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d435e71-f52d-4a3d-921a-8c5824bbcd02 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:07.326199 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:07.327128 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:07.327413 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:07.327893 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c admin admin] PATH_INFO: `/v3/projects/d21026bdfd1d47c2a30e97fce3fb70ae` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:07.329100 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d21026bdfd1d47c2a30e97fce3fb70ae)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:07.331295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:07.447970 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd21026bdfd1d47c2a30e97fce3fb70ae', 'request_id': 'req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:07.491848 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd21026bdfd1d47c2a30e97fce3fb70ae', 'request_id': 'req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:07.498816 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd21026bdfd1d47c2a30e97fce3fb70ae', 'request_id': 'req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:07.511790 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c admin admin] The token cache is being invalidate because project d21026bdfd1d47c2a30e97fce3fb70ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:29:07.512125 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:07.512859 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-abc65ce5-4596-43a5-b94a-b5a7d9149f5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:07.514601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1414/2839] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:29:07 2022] DELETE /identity/v3/projects/d21026bdfd1d47c2a30e97fce3fb70ae => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:29:07.521392 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355413d0-c386-4b65-9dbb-7d3a30565088 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:07.521982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355413d0-c386-4b65-9dbb-7d3a30565088 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:07.521982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355413d0-c386-4b65-9dbb-7d3a30565088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:07.543306 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-355413d0-c386-4b65-9dbb-7d3a30565088 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:07.544541 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-355413d0-c386-4b65-9dbb-7d3a30565088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:07.562960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:07.610176 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1426/2840] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:07.621486 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a012f6-346f-41c9-8051-8c6b08863151 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:07.622091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a012f6-346f-41c9-8051-8c6b08863151 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:07.622230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a012f6-346f-41c9-8051-8c6b08863151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:07.643810 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-06a012f6-346f-41c9-8051-8c6b08863151 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:07.645104 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-06a012f6-346f-41c9-8051-8c6b08863151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:07.653767 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1415/2841] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:29:07.658255 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355413d0-c386-4b65-9dbb-7d3a30565088 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:07.668999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355413d0-c386-4b65-9dbb-7d3a30565088 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:07.673656 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af50b6c-f1b1-41ed-a7be-53f3451253df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:07.674341 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af50b6c-f1b1-41ed-a7be-53f3451253df admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:07.674638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af50b6c-f1b1-41ed-a7be-53f3451253df admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:07.674919 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af50b6c-f1b1-41ed-a7be-53f3451253df admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:07.675652 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af50b6c-f1b1-41ed-a7be-53f3451253df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:29:07.675963 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af50b6c-f1b1-41ed-a7be-53f3451253df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:07.677591 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af50b6c-f1b1-41ed-a7be-53f3451253df admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:07.690281 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1427/2842] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:29:07.694822 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a012f6-346f-41c9-8051-8c6b08863151 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:07.704400 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a012f6-346f-41c9-8051-8c6b08863151 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:07.709040 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeaf9f5-79e8-4215-9cba-0355861c8336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:07.709701 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaf9f5-79e8-4215-9cba-0355861c8336 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:07.709987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaf9f5-79e8-4215-9cba-0355861c8336 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:07.710266 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaf9f5-79e8-4215-9cba-0355861c8336 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:07.710992 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeaf9f5-79e8-4215-9cba-0355861c8336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:07.712977 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeaf9f5-79e8-4215-9cba-0355861c8336 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:07.790985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1416/2843] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:29:07 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:29:07.796018 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af50b6c-f1b1-41ed-a7be-53f3451253df admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:07.808996 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af50b6c-f1b1-41ed-a7be-53f3451253df admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:07.871479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f23dbd-aa94-4ebc-af4f-93276385764f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:07.872142 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f23dbd-aa94-4ebc-af4f-93276385764f admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:07.872436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f23dbd-aa94-4ebc-af4f-93276385764f admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:07.872719 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f23dbd-aa94-4ebc-af4f-93276385764f admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:07.873418 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f23dbd-aa94-4ebc-af4f-93276385764f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:07.875391 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f23dbd-aa94-4ebc-af4f-93276385764f admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:07.967117 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1428/2844] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:29:07 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:29:07.971572 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeaf9f5-79e8-4215-9cba-0355861c8336 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:07.984795 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeaf9f5-79e8-4215-9cba-0355861c8336 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:07.989789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d2831-8e72-475c-8c10-23b6539b467a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:07.990844 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d2831-8e72-475c-8c10-23b6539b467a admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:07.991132 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d2831-8e72-475c-8c10-23b6539b467a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:07.991412 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d2831-8e72-475c-8c10-23b6539b467a admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:07.992142 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d2831-8e72-475c-8c10-23b6539b467a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:07.993754 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d2831-8e72-475c-8c10-23b6539b467a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:08.006097 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1417/2845] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:29:08.011013 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f23dbd-aa94-4ebc-af4f-93276385764f admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:08.021482 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f23dbd-aa94-4ebc-af4f-93276385764f admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:08.027316 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ccbc9-aaa7-45e3-8f34-a6636cbf147e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:08.027999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ccbc9-aaa7-45e3-8f34-a6636cbf147e admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:08.028291 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ccbc9-aaa7-45e3-8f34-a6636cbf147e admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:08.028572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ccbc9-aaa7-45e3-8f34-a6636cbf147e admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:08.029305 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ccbc9-aaa7-45e3-8f34-a6636cbf147e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:08.030977 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ccbc9-aaa7-45e3-8f34-a6636cbf147e admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:08.042431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1429/2846] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:29:08.046819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d2831-8e72-475c-8c10-23b6539b467a admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:08.058189 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d2831-8e72-475c-8c10-23b6539b467a admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:08.063324 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838c131-9b44-45a1-a226-db70e86cc924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:08.064251 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838c131-9b44-45a1-a226-db70e86cc924 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:08.064540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838c131-9b44-45a1-a226-db70e86cc924 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:08.064820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838c131-9b44-45a1-a226-db70e86cc924 admin admin] PATH_INFO: `/v3/projects/ce16719a27e24eaa8a8f77f33115e227/users/9fe36380cb9044e0acc1ab2ad7d4c1b3/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:08.082117 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838c131-9b44-45a1-a226-db70e86cc924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce16719a27e24eaa8a8f77f33115e227, user_id=9fe36380cb9044e0acc1ab2ad7d4c1b3, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:08.084847 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838c131-9b44-45a1-a226-db70e86cc924 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:08.174519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1418/2847] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:29:08 2022] PUT /identity/v3/projects/ce16719a27e24eaa8a8f77f33115e227/users/9fe36380cb9044e0acc1ab2ad7d4c1b3/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:29:08.186203 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6e088-4cca-4b8d-9bab-48db88d1db48 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:08.186532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6e088-4cca-4b8d-9bab-48db88d1db48 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:08.186816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6e088-4cca-4b8d-9bab-48db88d1db48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:08.216552 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-9da6e088-4cca-4b8d-9bab-48db88d1db48 None None] MFA Rules not processed for user `9fe36380cb9044e0acc1ab2ad7d4c1b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:08.217799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9da6e088-4cca-4b8d-9bab-48db88d1db48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:08.291579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1430/2848] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:29:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:08.302283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838c131-9b44-45a1-a226-db70e86cc924 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:08.304044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c838c131-9b44-45a1-a226-db70e86cc924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:08.334173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:08.374224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838c131-9b44-45a1-a226-db70e86cc924 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:08.378796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91929668-7171-460f-b1fe-c006160d1762 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:08.379458 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91929668-7171-460f-b1fe-c006160d1762 service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:08.379746 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91929668-7171-460f-b1fe-c006160d1762 service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:08.380030 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91929668-7171-460f-b1fe-c006160d1762 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:08.389579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91929668-7171-460f-b1fe-c006160d1762 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:08.396296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91929668-7171-460f-b1fe-c006160d1762 service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:08.400913 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1419/2849] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:29:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:29:10.349965 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fc3e7-9636-4a96-b6ab-9ce740c36457 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:10.350656 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fc3e7-9636-4a96-b6ab-9ce740c36457 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:10.350656 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fc3e7-9636-4a96-b6ab-9ce740c36457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:10.371865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-820fc3e7-9636-4a96-b6ab-9ce740c36457 None None] MFA Rules not processed for user `9fe36380cb9044e0acc1ab2ad7d4c1b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:10.373102 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-820fc3e7-9636-4a96-b6ab-9ce740c36457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:10.380578 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1431/2850] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:29:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:11.977211 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91929668-7171-460f-b1fe-c006160d1762 service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:11.978591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-91929668-7171-460f-b1fe-c006160d1762 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:12.057432 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91929668-7171-460f-b1fe-c006160d1762 service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:12.063670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba99007-1ac6-43f7-92fc-be16dc169971 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:12.064348 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba99007-1ac6-43f7-92fc-be16dc169971 service placement] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:12.064642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba99007-1ac6-43f7-92fc-be16dc169971 service placement] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:12.064956 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba99007-1ac6-43f7-92fc-be16dc169971 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:12.066793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-8ba99007-1ac6-43f7-92fc-be16dc169971 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:12.081472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba99007-1ac6-43f7-92fc-be16dc169971 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:12.084793 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba99007-1ac6-43f7-92fc-be16dc169971 service placement] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:12.095799 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1420/2851] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:29:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:29:20.366213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:22.089638 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=49 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:26.802436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820fc3e7-9636-4a96-b6ab-9ce740c36457 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:26.803763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-820fc3e7-9636-4a96-b6ab-9ce740c36457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:26.878644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820fc3e7-9636-4a96-b6ab-9ce740c36457 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:26.884278 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1a04f-718f-4847-b685-173eb4b18a87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:26.884926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1a04f-718f-4847-b685-173eb4b18a87 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:26.885220 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1a04f-718f-4847-b685-173eb4b18a87 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:26.886080 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1a04f-718f-4847-b685-173eb4b18a87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:26.887843 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1bf1a04f-718f-4847-b685-173eb4b18a87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:26.900717 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1a04f-718f-4847-b685-173eb4b18a87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:26.902375 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1a04f-718f-4847-b685-173eb4b18a87 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:26.913485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1432/2852] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:29:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:29:34.128794 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc3dac-ed3e-4466-ae2f-7be289d2c1d2 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:34.129366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc3dac-ed3e-4466-ae2f-7be289d2c1d2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:34.129366 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc3dac-ed3e-4466-ae2f-7be289d2c1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:34.196297 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-a3dc3dac-ed3e-4466-ae2f-7be289d2c1d2 None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:34.197533 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a3dc3dac-ed3e-4466-ae2f-7be289d2c1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:34.268895 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1421/2853] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:29:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:34.398705 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1a04f-718f-4847-b685-173eb4b18a87 service nova] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:34.400168 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1bf1a04f-718f-4847-b685-173eb4b18a87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:34.495968 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1a04f-718f-4847-b685-173eb4b18a87 service nova] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:34.501108 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429032c-b1b1-492c-837c-02748d5da152 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:34.501760 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429032c-b1b1-492c-837c-02748d5da152 service swift] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:34.502051 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429032c-b1b1-492c-837c-02748d5da152 service swift] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:34.502352 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429032c-b1b1-492c-837c-02748d5da152 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:34.511729 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429032c-b1b1-492c-837c-02748d5da152 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:34.513224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429032c-b1b1-492c-837c-02748d5da152 service swift] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:34.521718 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1433/2854] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:29:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 19:29:35.404983 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdb44e-7938-4d11-98c4-4e54d8711945 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:35.405591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdb44e-7938-4d11-98c4-4e54d8711945 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:35.405591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdb44e-7938-4d11-98c4-4e54d8711945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:35.426216 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-82bdb44e-7938-4d11-98c4-4e54d8711945 None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:35.427519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-82bdb44e-7938-4d11-98c4-4e54d8711945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:35.435311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1422/2855] 10.222.0.17 () {58 vars in 967 bytes} [Mon Jul 25 19:29:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:35.448063 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429032c-b1b1-492c-837c-02748d5da152 service swift] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:35.459469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429032c-b1b1-492c-837c-02748d5da152 service swift] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:35.465449 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a1802-d523-494d-bb4e-643dc28576b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:35.466028 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a1802-d523-494d-bb4e-643dc28576b6 service swift] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:35.466374 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a1802-d523-494d-bb4e-643dc28576b6 service swift] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:35.466676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a1802-d523-494d-bb4e-643dc28576b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:35.475453 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a1802-d523-494d-bb4e-643dc28576b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:35.476975 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a1802-d523-494d-bb4e-643dc28576b6 service swift] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:35.486395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1434/2856] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:29:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:29:36.131995 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2b8f7-f8a0-4512-9383-f6cc0da94066 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:36.132291 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2b8f7-f8a0-4512-9383-f6cc0da94066 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:36.132578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2b8f7-f8a0-4512-9383-f6cc0da94066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:36.153714 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-4cd2b8f7-f8a0-4512-9383-f6cc0da94066 None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:36.155003 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-4cd2b8f7-f8a0-4512-9383-f6cc0da94066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:36.162715 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1423/2857] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:36.175198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a1802-d523-494d-bb4e-643dc28576b6 service swift] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:36.186457 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a1802-d523-494d-bb4e-643dc28576b6 service swift] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:36.191905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d8ff0-fe9f-41f0-a10b-07accd2454d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:36.192573 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d8ff0-fe9f-41f0-a10b-07accd2454d6 service swift] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:36.192865 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d8ff0-fe9f-41f0-a10b-07accd2454d6 service swift] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:36.193139 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d8ff0-fe9f-41f0-a10b-07accd2454d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:36.202479 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d8ff0-fe9f-41f0-a10b-07accd2454d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:36.204076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d8ff0-fe9f-41f0-a10b-07accd2454d6 service swift] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:36.213024 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1435/2858] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:29:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:29:36.815405 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259ad66-fccb-46a9-a2fa-d8a5951cdb4d None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:36.815897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259ad66-fccb-46a9-a2fa-d8a5951cdb4d None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:36.816011 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259ad66-fccb-46a9-a2fa-d8a5951cdb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:36.836204 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-a259ad66-fccb-46a9-a2fa-d8a5951cdb4d None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:36.837435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a259ad66-fccb-46a9-a2fa-d8a5951cdb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:36.845607 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1424/2859] 10.222.0.17 () {58 vars in 967 bytes} [Mon Jul 25 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:36.859049 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d8ff0-fe9f-41f0-a10b-07accd2454d6 service swift] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:36.870027 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d8ff0-fe9f-41f0-a10b-07accd2454d6 service swift] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:36.875314 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302abbea-2824-45d1-8aac-2f13aedc7845 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:36.875967 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302abbea-2824-45d1-8aac-2f13aedc7845 service swift] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:36.876257 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302abbea-2824-45d1-8aac-2f13aedc7845 service swift] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:36.876577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302abbea-2824-45d1-8aac-2f13aedc7845 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:36.885844 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302abbea-2824-45d1-8aac-2f13aedc7845 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:36.888302 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302abbea-2824-45d1-8aac-2f13aedc7845 service swift] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:36.897406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1436/2860] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:29:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:29:39.624897 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259ad66-fccb-46a9-a2fa-d8a5951cdb4d None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:39.693919 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259ad66-fccb-46a9-a2fa-d8a5951cdb4d None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:39.699431 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:39.700416 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:39.700707 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:39.700991 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] PATH_INFO: `/v3/users/9fe36380cb9044e0acc1ab2ad7d4c1b3` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:39.702408 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fe36380cb9044e0acc1ab2ad7d4c1b3)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:39.704396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:39.884921 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:39.886712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fe36380cb9044e0acc1ab2ad7d4c1b3'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:39.887394 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fe36380cb9044e0acc1ab2ad7d4c1b3'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:39.888091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fe36380cb9044e0acc1ab2ad7d4c1b3'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:39.967466 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fe36380cb9044e0acc1ab2ad7d4c1b3'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:39.985528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fe36380cb9044e0acc1ab2ad7d4c1b3'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:39.999240 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1425/2861] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:29:39 2022] DELETE /identity/v3/users/9fe36380cb9044e0acc1ab2ad7d4c1b3 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:29:40.004138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d6235-07d9-4710-997e-cbeefe8dea1f None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:40.004628 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d6235-07d9-4710-997e-cbeefe8dea1f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:40.004756 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d6235-07d9-4710-997e-cbeefe8dea1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:40.025820 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-284d6235-07d9-4710-997e-cbeefe8dea1f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:40.027138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-284d6235-07d9-4710-997e-cbeefe8dea1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:40.041900 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=100 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:40.095205 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1437/2862] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:40.106667 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:40.108042 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:40.169158 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa2112-8b86-4659-8073-941bf920cae1 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:40.173165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019073a2-9427-418b-8540-9a2524aa5988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:40.173825 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019073a2-9427-418b-8540-9a2524aa5988 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:40.174112 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019073a2-9427-418b-8540-9a2524aa5988 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:40.174426 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019073a2-9427-418b-8540-9a2524aa5988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:40.184647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019073a2-9427-418b-8540-9a2524aa5988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:40.186574 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019073a2-9427-418b-8540-9a2524aa5988 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:40.196575 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1426/2863] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:29:40.460077 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d6235-07d9-4710-997e-cbeefe8dea1f None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:40.461650 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-284d6235-07d9-4710-997e-cbeefe8dea1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:40.475046 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d6235-07d9-4710-997e-cbeefe8dea1f None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:40.479316 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:40.480195 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:40.480475 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:40.480750 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b admin admin] PATH_INFO: `/v3/projects/ce16719a27e24eaa8a8f77f33115e227` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:40.481993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce16719a27e24eaa8a8f77f33115e227)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:40.484068 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:40.614674 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce16719a27e24eaa8a8f77f33115e227', 'request_id': 'req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:40.625216 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce16719a27e24eaa8a8f77f33115e227', 'request_id': 'req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:40.635678 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce16719a27e24eaa8a8f77f33115e227', 'request_id': 'req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:40.707790 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b admin admin] The token cache is being invalidate because project ce16719a27e24eaa8a8f77f33115e227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86905) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:29:40.708076 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:40.708753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-70a01b67-c3da-4e1d-be2f-b84bb5a1044b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:29:40.710395 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1438/2864] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:29:40 2022] DELETE /identity/v3/projects/ce16719a27e24eaa8a8f77f33115e227 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:29:40.715438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79617c7-17ed-4517-accf-eda916f400c7 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:40.716332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79617c7-17ed-4517-accf-eda916f400c7 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:40.716332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79617c7-17ed-4517-accf-eda916f400c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:40.737433 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-d79617c7-17ed-4517-accf-eda916f400c7 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:40.738715 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d79617c7-17ed-4517-accf-eda916f400c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:40.809487 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1427/2865] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:40.817802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a91f34-771e-4356-b530-a632476a798f None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:40.818088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a91f34-771e-4356-b530-a632476a798f None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:40.818403 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a91f34-771e-4356-b530-a632476a798f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:40.839179 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-c4a91f34-771e-4356-b530-a632476a798f None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:40.840404 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c4a91f34-771e-4356-b530-a632476a798f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:40.849783 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1439/2866] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:40.854374 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79617c7-17ed-4517-accf-eda916f400c7 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:40.864725 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79617c7-17ed-4517-accf-eda916f400c7 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:40.869356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a5282-f4b6-4ed0-91a3-0e3e1b78d839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:40.870028 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5282-f4b6-4ed0-91a3-0e3e1b78d839 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:40.870316 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5282-f4b6-4ed0-91a3-0e3e1b78d839 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:40.870621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5282-f4b6-4ed0-91a3-0e3e1b78d839 admin admin] PATH_INFO: `/v3/domains` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:40.871382 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a5282-f4b6-4ed0-91a3-0e3e1b78d839 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86906) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:29:40.871692 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a5282-f4b6-4ed0-91a3-0e3e1b78d839 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:40.873590 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a5282-f4b6-4ed0-91a3-0e3e1b78d839 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:40.886296 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1428/2867] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:29:40.891417 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a91f34-771e-4356-b530-a632476a798f None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:40.901539 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a91f34-771e-4356-b530-a632476a798f None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:40.906970 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff541702-7bbf-481d-a363-1015cd798e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:40.907635 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff541702-7bbf-481d-a363-1015cd798e48 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:40.907922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff541702-7bbf-481d-a363-1015cd798e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:40.908231 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff541702-7bbf-481d-a363-1015cd798e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:40.908931 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff541702-7bbf-481d-a363-1015cd798e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:40.910855 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff541702-7bbf-481d-a363-1015cd798e48 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:41.044334 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1440/2868] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:29:40 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:29:41.048765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a5282-f4b6-4ed0-91a3-0e3e1b78d839 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:41.059249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a5282-f4b6-4ed0-91a3-0e3e1b78d839 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:41.108167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:41.117312 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd4fc3-0878-43f2-83d6-4856757c6f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:41.118010 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4fc3-0878-43f2-83d6-4856757c6f51 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:41.118284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4fc3-0878-43f2-83d6-4856757c6f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:41.118604 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4fc3-0878-43f2-83d6-4856757c6f51 admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:41.119306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd4fc3-0878-43f2-83d6-4856757c6f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:41.121231 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd4fc3-0878-43f2-83d6-4856757c6f51 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:41.212268 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1429/2869] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:29:41 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:29:41.216922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff541702-7bbf-481d-a363-1015cd798e48 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:41.229239 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff541702-7bbf-481d-a363-1015cd798e48 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:41.234524 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231a05d-2b80-4705-92f0-70bc23773d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:41.235188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231a05d-2b80-4705-92f0-70bc23773d90 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:41.235471 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231a05d-2b80-4705-92f0-70bc23773d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:41.235752 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231a05d-2b80-4705-92f0-70bc23773d90 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:41.236537 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231a05d-2b80-4705-92f0-70bc23773d90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:41.238121 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231a05d-2b80-4705-92f0-70bc23773d90 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:41.249408 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1441/2870] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:29:41.253493 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd4fc3-0878-43f2-83d6-4856757c6f51 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:41.265141 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd4fc3-0878-43f2-83d6-4856757c6f51 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:41.269863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26ba51-19e7-4c8d-afaf-64d2e060e14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:41.270568 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26ba51-19e7-4c8d-afaf-64d2e060e14a admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:41.270864 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26ba51-19e7-4c8d-afaf-64d2e060e14a admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:41.271147 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26ba51-19e7-4c8d-afaf-64d2e060e14a admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:41.271876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26ba51-19e7-4c8d-afaf-64d2e060e14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:41.273484 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26ba51-19e7-4c8d-afaf-64d2e060e14a admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:41.284942 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1430/2871] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:29:41.289187 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231a05d-2b80-4705-92f0-70bc23773d90 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:41.299306 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231a05d-2b80-4705-92f0-70bc23773d90 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:41.303813 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d39761-f02f-444b-93b8-501a19107088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:41.306240 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d39761-f02f-444b-93b8-501a19107088 admin admin] REQUEST_METHOD: `PUT` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:41.306553 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d39761-f02f-444b-93b8-501a19107088 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:41.306829 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d39761-f02f-444b-93b8-501a19107088 admin admin] PATH_INFO: `/v3/projects/c31a209e911b44eebde1216a909de48a/users/4df91b4cddab45f2ac7686e7c9fd00be/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:41.321981 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d39761-f02f-444b-93b8-501a19107088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31a209e911b44eebde1216a909de48a, user_id=4df91b4cddab45f2ac7686e7c9fd00be, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:41.324788 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d39761-f02f-444b-93b8-501a19107088 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:41.420450 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1442/2872] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:29:41 2022] PUT /identity/v3/projects/c31a209e911b44eebde1216a909de48a/users/4df91b4cddab45f2ac7686e7c9fd00be/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:29:41.426022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e70b2e-338e-4277-9ae8-8751a829d5ea None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:41.426311 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e70b2e-338e-4277-9ae8-8751a829d5ea None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:41.426627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e70b2e-338e-4277-9ae8-8751a829d5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:41.447295 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-c6e70b2e-338e-4277-9ae8-8751a829d5ea None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:41.448535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c6e70b2e-338e-4277-9ae8-8751a829d5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:41.516125 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1431/2873] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:41.526073 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d39761-f02f-444b-93b8-501a19107088 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:41.527966 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-98d39761-f02f-444b-93b8-501a19107088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:41.573945 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:41.599964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d39761-f02f-444b-93b8-501a19107088 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:41.606214 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2551c49-e299-4cab-bda3-8a482bb261c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:41.607921 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2551c49-e299-4cab-bda3-8a482bb261c6 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:41.608409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2551c49-e299-4cab-bda3-8a482bb261c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:41.609178 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2551c49-e299-4cab-bda3-8a482bb261c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:41.619310 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2551c49-e299-4cab-bda3-8a482bb261c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:41.620757 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2551c49-e299-4cab-bda3-8a482bb261c6 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:41.631833 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1443/2874] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:29:47.354363 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a556b1-b36f-439a-a2a5-edb8f03ecdae None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:47.354929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a556b1-b36f-439a-a2a5-edb8f03ecdae None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:47.354929 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a556b1-b36f-439a-a2a5-edb8f03ecdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:47.389873 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-e9a556b1-b36f-439a-a2a5-edb8f03ecdae None None] MFA Rules not processed for user `4df91b4cddab45f2ac7686e7c9fd00be`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:47.391181 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-e9a556b1-b36f-439a-a2a5-edb8f03ecdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:47.483819 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1432/2875] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:29:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:47.485188 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2551c49-e299-4cab-bda3-8a482bb261c6 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:47.495868 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2551c49-e299-4cab-bda3-8a482bb261c6 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:47.502259 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d33d8-b3d3-4c8b-84db-514aad53d7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:47.503088 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d33d8-b3d3-4c8b-84db-514aad53d7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:47.503449 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d33d8-b3d3-4c8b-84db-514aad53d7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:47.504189 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d33d8-b3d3-4c8b-84db-514aad53d7b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:47.504964 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d33d8-b3d3-4c8b-84db-514aad53d7b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:47.507015 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d33d8-b3d3-4c8b-84db-514aad53d7b0 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:47.600616 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1444/2876] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:29:47 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:29:47.601908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a556b1-b36f-439a-a2a5-edb8f03ecdae None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:47.616591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a556b1-b36f-439a-a2a5-edb8f03ecdae None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:47.689625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d222b-aff9-48ad-97b0-e12e9b0d12ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:47.690372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d222b-aff9-48ad-97b0-e12e9b0d12ac admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:47.690647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d222b-aff9-48ad-97b0-e12e9b0d12ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:47.690936 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d222b-aff9-48ad-97b0-e12e9b0d12ac admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:47.691670 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d222b-aff9-48ad-97b0-e12e9b0d12ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:47.693623 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d222b-aff9-48ad-97b0-e12e9b0d12ac admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:47.703314 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:47.805376 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1433/2877] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:29:47 2022] POST /identity/v3/users => generated 298 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:29:47.809758 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d33d8-b3d3-4c8b-84db-514aad53d7b0 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:47.824768 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d33d8-b3d3-4c8b-84db-514aad53d7b0 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:47.826821 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d5028-331e-4432-963c-a864515ca789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:47.827570 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d5028-331e-4432-963c-a864515ca789 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:47.827856 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d5028-331e-4432-963c-a864515ca789 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:47.828138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d5028-331e-4432-963c-a864515ca789 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:47.828869 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d5028-331e-4432-963c-a864515ca789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:47.831470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d5028-331e-4432-963c-a864515ca789 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:47.846802 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1445/2878] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:29:47.851040 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d222b-aff9-48ad-97b0-e12e9b0d12ac admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:47.862614 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d222b-aff9-48ad-97b0-e12e9b0d12ac admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:47.867457 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0c519-5213-425e-9663-dc342ff86069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:47.868375 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0c519-5213-425e-9663-dc342ff86069 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:47.868665 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0c519-5213-425e-9663-dc342ff86069 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:47.868948 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0c519-5213-425e-9663-dc342ff86069 admin admin] PATH_INFO: `/v3/projects/1a99d4e0d3d44e5b8103b8a51ddd7d49/users/16ba2db446f44ed58e812f2ff4ee6d02/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:47.888522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0c519-5213-425e-9663-dc342ff86069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a99d4e0d3d44e5b8103b8a51ddd7d49, user_id=16ba2db446f44ed58e812f2ff4ee6d02, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:47.891280 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0c519-5213-425e-9663-dc342ff86069 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:47.961071 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1434/2879] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:29:47 2022] PUT /identity/v3/projects/1a99d4e0d3d44e5b8103b8a51ddd7d49/users/16ba2db446f44ed58e812f2ff4ee6d02/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:29:47.967620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d5028-331e-4432-963c-a864515ca789 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:47.979209 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d5028-331e-4432-963c-a864515ca789 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:48.039696 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99843834-c144-44ef-8ff5-c975bbc10830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:48.040177 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99843834-c144-44ef-8ff5-c975bbc10830 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:48.040533 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99843834-c144-44ef-8ff5-c975bbc10830 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:48.040747 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99843834-c144-44ef-8ff5-c975bbc10830 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:48.041483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99843834-c144-44ef-8ff5-c975bbc10830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:48.043250 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99843834-c144-44ef-8ff5-c975bbc10830 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:48.055032 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1446/2880] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:29:48.058905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0c519-5213-425e-9663-dc342ff86069 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:48.070700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0c519-5213-425e-9663-dc342ff86069 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:48.076219 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0879473-9fca-40cd-97b4-37781b37101e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:48.077332 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0879473-9fca-40cd-97b4-37781b37101e admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:48.077621 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0879473-9fca-40cd-97b4-37781b37101e admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:48.077908 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0879473-9fca-40cd-97b4-37781b37101e admin admin] PATH_INFO: `/v3/projects/1a99d4e0d3d44e5b8103b8a51ddd7d49/users/16ba2db446f44ed58e812f2ff4ee6d02/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:48.080022 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0879473-9fca-40cd-97b4-37781b37101e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a99d4e0d3d44e5b8103b8a51ddd7d49, user_id=16ba2db446f44ed58e812f2ff4ee6d02, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:48.082790 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0879473-9fca-40cd-97b4-37781b37101e admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:48.096085 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1435/2881] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:29:48 2022] PUT /identity/v3/projects/1a99d4e0d3d44e5b8103b8a51ddd7d49/users/16ba2db446f44ed58e812f2ff4ee6d02/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:29:54.119106 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f98ea3-eca2-4587-a760-355a8c726b51 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:54.120943 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f98ea3-eca2-4587-a760-355a8c726b51 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:54.120943 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f98ea3-eca2-4587-a760-355a8c726b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:54.155515 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-60f98ea3-eca2-4587-a760-355a8c726b51 None None] MFA Rules not processed for user `16ba2db446f44ed58e812f2ff4ee6d02`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:54.157092 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-60f98ea3-eca2-4587-a760-355a8c726b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:54.236059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1447/2882] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:29:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:29:54.250586 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0879473-9fca-40cd-97b4-37781b37101e admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:54.254093 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d0879473-9fca-40cd-97b4-37781b37101e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:54.321725 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0879473-9fca-40cd-97b4-37781b37101e admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:54.325648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d405b4-fb65-4535-8761-4f281c7a8beb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28954d67689a46ee975fdfc9139e41eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:54.326557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d405b4-fb65-4535-8761-4f281c7a8beb service glance] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:54.326935 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d405b4-fb65-4535-8761-4f281c7a8beb service glance] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:54.327225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d405b4-fb65-4535-8761-4f281c7a8beb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:54.411481 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d405b4-fb65-4535-8761-4f281c7a8beb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:54.412883 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d405b4-fb65-4535-8761-4f281c7a8beb service glance] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:54.422795 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1436/2883] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:29:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:29:54.669148 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85594e-549d-457d-9d38-a13805f4fd88 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:54.669467 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85594e-549d-457d-9d38-a13805f4fd88 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:54.669749 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85594e-549d-457d-9d38-a13805f4fd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:54.689939 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-aa85594e-549d-457d-9d38-a13805f4fd88 None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:54.691176 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-aa85594e-549d-457d-9d38-a13805f4fd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:54.713101 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:54.753002 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1448/2884] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:29:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:54.766739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d405b4-fb65-4535-8761-4f281c7a8beb service glance] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:54.768284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b3d405b4-fb65-4535-8761-4f281c7a8beb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:54.875495 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d405b4-fb65-4535-8761-4f281c7a8beb service glance] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:54.885832 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad776e-bdae-4f1e-9b56-98e243ef592a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:54.886642 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad776e-bdae-4f1e-9b56-98e243ef592a service swift] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:54.886826 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad776e-bdae-4f1e-9b56-98e243ef592a service swift] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:54.887092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad776e-bdae-4f1e-9b56-98e243ef592a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:54.897357 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad776e-bdae-4f1e-9b56-98e243ef592a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:54.899019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad776e-bdae-4f1e-9b56-98e243ef592a service swift] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:54.903212 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:54.909196 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1437/2885] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:29:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 19:30:04.776887 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=13 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:30:04.994111 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:30:06.263014 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480df7d-339a-408c-91b2-74d813449a04 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:30:06.263736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480df7d-339a-408c-91b2-74d813449a04 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:30:06.263736 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480df7d-339a-408c-91b2-74d813449a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:30:06.283593 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-6480df7d-339a-408c-91b2-74d813449a04 None None] MFA Rules not processed for user `4df91b4cddab45f2ac7686e7c9fd00be`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:30:06.284926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6480df7d-339a-408c-91b2-74d813449a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:30:06.292513 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1449/2886] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:30:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:30:06.594817 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad776e-bdae-4f1e-9b56-98e243ef592a service swift] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:30:06.656905 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad776e-bdae-4f1e-9b56-98e243ef592a service swift] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:30:06.661130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214259a5-256c-4c0c-b839-a2f5b84ee328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:30:06.661885 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214259a5-256c-4c0c-b839-a2f5b84ee328 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:30:06.662228 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214259a5-256c-4c0c-b839-a2f5b84ee328 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:30:06.662712 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214259a5-256c-4c0c-b839-a2f5b84ee328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:30:06.664820 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-214259a5-256c-4c0c-b839-a2f5b84ee328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:30:06.677780 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214259a5-256c-4c0c-b839-a2f5b84ee328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:30:06.679518 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214259a5-256c-4c0c-b839-a2f5b84ee328 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:30:06.689635 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1438/2887] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:30:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:30:09.882120 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154966f-8ff2-4b08-820d-336faddb43f3 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:30:09.882687 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154966f-8ff2-4b08-820d-336faddb43f3 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:30:09.883032 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154966f-8ff2-4b08-820d-336faddb43f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:30:09.903681 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-a154966f-8ff2-4b08-820d-336faddb43f3 None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:30:09.904900 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a154966f-8ff2-4b08-820d-336faddb43f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:30:09.912907 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1450/2888] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 19:30:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:30:09.949677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214259a5-256c-4c0c-b839-a2f5b84ee328 service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:30:09.949677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214259a5-256c-4c0c-b839-a2f5b84ee328 service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:30:09.949677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76947db2-507e-466b-9b3d-8a632f41bc4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:30:09.949677 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76947db2-507e-466b-9b3d-8a632f41bc4f service swift] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:30:09.951229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76947db2-507e-466b-9b3d-8a632f41bc4f service swift] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:30:09.951229 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76947db2-507e-466b-9b3d-8a632f41bc4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:30:09.965702 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76947db2-507e-466b-9b3d-8a632f41bc4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:30:09.968585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76947db2-507e-466b-9b3d-8a632f41bc4f service swift] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:30:09.979303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1439/2889] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:30:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:30:19.898064 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=6 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:30:19.974727 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=32 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:30:21.557685 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154966f-8ff2-4b08-820d-336faddb43f3 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:30:21.559948 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-a154966f-8ff2-4b08-820d-336faddb43f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:30:21.631496 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154966f-8ff2-4b08-820d-336faddb43f3 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:30:21.635436 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ec47a-fc24-4cd9-8729-14eea3691417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:30:21.636100 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ec47a-fc24-4cd9-8729-14eea3691417 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:30:21.636388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ec47a-fc24-4cd9-8729-14eea3691417 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:30:21.636670 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ec47a-fc24-4cd9-8729-14eea3691417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:30:21.639605 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-429ec47a-fc24-4cd9-8729-14eea3691417 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:30:21.650620 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ec47a-fc24-4cd9-8729-14eea3691417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:30:21.651923 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ec47a-fc24-4cd9-8729-14eea3691417 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:30:21.662570 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1451/2890] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:30:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:30:31.657978 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=28 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:30:44.443686 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76947db2-507e-466b-9b3d-8a632f41bc4f service swift] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:30:44.464941 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76947db2-507e-466b-9b3d-8a632f41bc4f service swift] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:30:44.468686 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205281b-08df-4958-87c4-6c64fa46f50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:30:44.469343 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205281b-08df-4958-87c4-6c64fa46f50a service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:30:44.469625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205281b-08df-4958-87c4-6c64fa46f50a service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:30:44.472333 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205281b-08df-4958-87c4-6c64fa46f50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:30:44.502412 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205281b-08df-4958-87c4-6c64fa46f50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:30:44.503860 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205281b-08df-4958-87c4-6c64fa46f50a service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:30:44.514972 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1440/2891] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:30:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:30:46.799659 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ec47a-fc24-4cd9-8729-14eea3691417 service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:30:46.805390 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-429ec47a-fc24-4cd9-8729-14eea3691417 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:30:46.902006 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ec47a-fc24-4cd9-8729-14eea3691417 service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:30:46.909212 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650245cc-15b3-438b-a6e5-0bfb6de3c5c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:30:46.909896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650245cc-15b3-438b-a6e5-0bfb6de3c5c8 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:30:46.910181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650245cc-15b3-438b-a6e5-0bfb6de3c5c8 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:30:46.910483 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650245cc-15b3-438b-a6e5-0bfb6de3c5c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:30:46.912737 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-650245cc-15b3-438b-a6e5-0bfb6de3c5c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:30:46.940233 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650245cc-15b3-438b-a6e5-0bfb6de3c5c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:30:46.941953 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650245cc-15b3-438b-a6e5-0bfb6de3c5c8 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:30:46.957466 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1452/2892] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:30:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:30:54.509779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=20 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:30:56.951799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=34 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:31:49.658601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205281b-08df-4958-87c4-6c64fa46f50a service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:31:49.668684 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205281b-08df-4958-87c4-6c64fa46f50a service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:31:49.673655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea821f31-f456-482a-9990-5d4e7d2dddca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:31:49.673655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea821f31-f456-482a-9990-5d4e7d2dddca service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:31:49.673655 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea821f31-f456-482a-9990-5d4e7d2dddca service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:31:49.676662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea821f31-f456-482a-9990-5d4e7d2dddca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:31:49.676662 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ea821f31-f456-482a-9990-5d4e7d2dddca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:31:49.691283 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea821f31-f456-482a-9990-5d4e7d2dddca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:31:49.692776 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea821f31-f456-482a-9990-5d4e7d2dddca service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:31:49.705499 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1441/2893] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:31:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:31:59.699561 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=6 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:33:10.085610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650245cc-15b3-438b-a6e5-0bfb6de3c5c8 service nova] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:10.095905 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650245cc-15b3-438b-a6e5-0bfb6de3c5c8 service nova] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:10.099993 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4fe72a-96ca-4fa5-b454-20dd97d77e42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:10.100638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fe72a-96ca-4fa5-b454-20dd97d77e42 service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:10.100922 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fe72a-96ca-4fa5-b454-20dd97d77e42 service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:10.101220 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fe72a-96ca-4fa5-b454-20dd97d77e42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:10.102990 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9e4fe72a-96ca-4fa5-b454-20dd97d77e42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:10.117891 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4fe72a-96ca-4fa5-b454-20dd97d77e42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:10.119322 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4fe72a-96ca-4fa5-b454-20dd97d77e42 service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:10.130809 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1453/2894] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:33:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:33:19.689567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea821f31-f456-482a-9990-5d4e7d2dddca service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:19.691052 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-ea821f31-f456-482a-9990-5d4e7d2dddca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:19.705456 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea821f31-f456-482a-9990-5d4e7d2dddca service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:19.711469 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903acbc6-e432-4792-8372-8cd0f3d503c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:19.716353 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903acbc6-e432-4792-8372-8cd0f3d503c5 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:19.716353 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903acbc6-e432-4792-8372-8cd0f3d503c5 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:19.716353 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903acbc6-e432-4792-8372-8cd0f3d503c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:19.716353 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-903acbc6-e432-4792-8372-8cd0f3d503c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:19.732903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903acbc6-e432-4792-8372-8cd0f3d503c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:19.735483 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903acbc6-e432-4792-8372-8cd0f3d503c5 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:19.748586 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1442/2895] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:33:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:33:20.125400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=6 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:33:20.944065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa43661-92ed-45d3-b275-8cef99cc81a8 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:20.944065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa43661-92ed-45d3-b275-8cef99cc81a8 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:20.944065 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa43661-92ed-45d3-b275-8cef99cc81a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:20.963416 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-caa43661-92ed-45d3-b275-8cef99cc81a8 None None] MFA Rules not processed for user `c6a88af6e45549f1ab00036923f6ba04`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:33:20.964796 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-caa43661-92ed-45d3-b275-8cef99cc81a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:20.972114 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1454/2896] 10.222.0.17 () {58 vars in 967 bytes} [Mon Jul 25 19:33:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:33:21.013537 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903acbc6-e432-4792-8372-8cd0f3d503c5 service nova] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:21.026230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903acbc6-e432-4792-8372-8cd0f3d503c5 service nova] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:21.031293 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93deda09-a009-4d2f-9253-9615192435bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ee9152572224cc499229e0fa2b81ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:21.031934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deda09-a009-4d2f-9253-9615192435bc service swift] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:21.032226 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deda09-a009-4d2f-9253-9615192435bc service swift] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:21.032505 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deda09-a009-4d2f-9253-9615192435bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:21.041206 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93deda09-a009-4d2f-9253-9615192435bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:21.042827 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93deda09-a009-4d2f-9253-9615192435bc service swift] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:21.051700 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1443/2897] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 19:33:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:33:25.465936 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa43661-92ed-45d3-b275-8cef99cc81a8 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:25.467506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-caa43661-92ed-45d3-b275-8cef99cc81a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:25.568348 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa43661-92ed-45d3-b275-8cef99cc81a8 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:25.573565 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aeb1e0-c25b-4846-9513-47f5d8671732 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:25.574227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeb1e0-c25b-4846-9513-47f5d8671732 service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:25.574572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeb1e0-c25b-4846-9513-47f5d8671732 service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:25.574860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeb1e0-c25b-4846-9513-47f5d8671732 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:25.576679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-43aeb1e0-c25b-4846-9513-47f5d8671732 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:25.589447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aeb1e0-c25b-4846-9513-47f5d8671732 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:25.591079 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aeb1e0-c25b-4846-9513-47f5d8671732 service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:25.601544 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1455/2898] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:33:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:33:29.294480 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93deda09-a009-4d2f-9253-9615192435bc service swift] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:29.307567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93deda09-a009-4d2f-9253-9615192435bc service swift] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:29.312888 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd57b04-abbb-4989-a667-e35c7b50aa78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:29.313559 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd57b04-abbb-4989-a667-e35c7b50aa78 service placement] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:29.313854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd57b04-abbb-4989-a667-e35c7b50aa78 service placement] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:29.314137 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd57b04-abbb-4989-a667-e35c7b50aa78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:29.316066 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-5dd57b04-abbb-4989-a667-e35c7b50aa78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:29.329472 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd57b04-abbb-4989-a667-e35c7b50aa78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:29.331092 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd57b04-abbb-4989-a667-e35c7b50aa78 service placement] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:29.342488 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1444/2899] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:33:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:33:29.411433 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aeb1e0-c25b-4846-9513-47f5d8671732 service placement] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:29.497227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aeb1e0-c25b-4846-9513-47f5d8671732 service placement] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:29.501985 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:29.502896 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:29.503185 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:29.503470 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] PATH_INFO: `/v3/users/4df91b4cddab45f2ac7686e7c9fd00be` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:29.504660 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4df91b4cddab45f2ac7686e7c9fd00be)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:29.507505 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:29.709707 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4df91b4cddab45f2ac7686e7c9fd00be'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:29.710402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4df91b4cddab45f2ac7686e7c9fd00be'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:29.711691 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:33:29.718410 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4df91b4cddab45f2ac7686e7c9fd00be'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:29.730837 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4df91b4cddab45f2ac7686e7c9fd00be'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:29.792535 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4df91b4cddab45f2ac7686e7c9fd00be'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:29.799200 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1456/2900] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:33:29 2022] DELETE /identity/v3/users/4df91b4cddab45f2ac7686e7c9fd00be => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:33:30.029067 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd57b04-abbb-4989-a667-e35c7b50aa78 service placement] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:30.043271 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd57b04-abbb-4989-a667-e35c7b50aa78 service placement] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:30.109172 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:30.113214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:30.113214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:30.113214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] PATH_INFO: `/v3/projects/c31a209e911b44eebde1216a909de48a` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:30.113214 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31a209e911b44eebde1216a909de48a)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:30.113805 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:30.241497 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31a209e911b44eebde1216a909de48a', 'request_id': 'req-35bdd04b-7765-46a5-994f-9f908c6f9b34'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:30.312789 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31a209e911b44eebde1216a909de48a', 'request_id': 'req-35bdd04b-7765-46a5-994f-9f908c6f9b34'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:30.319417 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31a209e911b44eebde1216a909de48a', 'request_id': 'req-35bdd04b-7765-46a5-994f-9f908c6f9b34'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:30.344217 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] The token cache is being invalidate because project c31a209e911b44eebde1216a909de48a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:33:30.345699 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:30.345699 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:30.346915 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1445/2901] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:33:30 2022] DELETE /identity/v3/projects/c31a209e911b44eebde1216a909de48a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:33:30.351916 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:30.353332 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:30.366882 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321cc94-acf6-430c-97b9-eaacd11a6761 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:30.427698 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7808f82-a73a-47a9-a9b1-732b7cbcb6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:30.428611 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7808f82-a73a-47a9-a9b1-732b7cbcb6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:30.428755 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7808f82-a73a-47a9-a9b1-732b7cbcb6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:30.429035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7808f82-a73a-47a9-a9b1-732b7cbcb6d3 admin admin] PATH_INFO: `/v3/users/16ba2db446f44ed58e812f2ff4ee6d02` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:30.430400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7808f82-a73a-47a9-a9b1-732b7cbcb6d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16ba2db446f44ed58e812f2ff4ee6d02)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:30.433753 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7808f82-a73a-47a9-a9b1-732b7cbcb6d3 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:30.699146 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b7808f82-a73a-47a9-a9b1-732b7cbcb6d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16ba2db446f44ed58e812f2ff4ee6d02'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:30.699839 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b7808f82-a73a-47a9-a9b1-732b7cbcb6d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16ba2db446f44ed58e812f2ff4ee6d02'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:30.707485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b7808f82-a73a-47a9-a9b1-732b7cbcb6d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16ba2db446f44ed58e812f2ff4ee6d02'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:30.719951 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b7808f82-a73a-47a9-a9b1-732b7cbcb6d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16ba2db446f44ed58e812f2ff4ee6d02'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:30.798538 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-b7808f82-a73a-47a9-a9b1-732b7cbcb6d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16ba2db446f44ed58e812f2ff4ee6d02'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:30.807473 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1457/2902] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:33:30 2022] DELETE /identity/v3/users/16ba2db446f44ed58e812f2ff4ee6d02 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:33:31.051624 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:31.054094 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:31.068825 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdd04b-7765-46a5-994f-9f908c6f9b34 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:31.110979 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:33:31.131174 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e9eec-6e34-4452-a885-dadf4805d605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:31.132137 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e9eec-6e34-4452-a885-dadf4805d605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:31.132429 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e9eec-6e34-4452-a885-dadf4805d605 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:31.132711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e9eec-6e34-4452-a885-dadf4805d605 admin admin] PATH_INFO: `/v3/projects/1a99d4e0d3d44e5b8103b8a51ddd7d49` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:31.134019 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e9eec-6e34-4452-a885-dadf4805d605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a99d4e0d3d44e5b8103b8a51ddd7d49)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:31.135993 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e9eec-6e34-4452-a885-dadf4805d605 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:31.310452 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-400e9eec-6e34-4452-a885-dadf4805d605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a99d4e0d3d44e5b8103b8a51ddd7d49', 'request_id': 'req-400e9eec-6e34-4452-a885-dadf4805d605'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:31.393372 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-400e9eec-6e34-4452-a885-dadf4805d605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a99d4e0d3d44e5b8103b8a51ddd7d49', 'request_id': 'req-400e9eec-6e34-4452-a885-dadf4805d605'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:31.399796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-400e9eec-6e34-4452-a885-dadf4805d605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a99d4e0d3d44e5b8103b8a51ddd7d49', 'request_id': 'req-400e9eec-6e34-4452-a885-dadf4805d605'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:31.412837 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-400e9eec-6e34-4452-a885-dadf4805d605 admin admin] The token cache is being invalidate because project 1a99d4e0d3d44e5b8103b8a51ddd7d49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:33:31.413130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-400e9eec-6e34-4452-a885-dadf4805d605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:31.413930 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-400e9eec-6e34-4452-a885-dadf4805d605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:33:31.415519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1446/2903] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:33:31 2022] DELETE /identity/v3/projects/1a99d4e0d3d44e5b8103b8a51ddd7d49 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:33:31.420741 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4e5fc-518b-4004-a3e8-73b393732ce2 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:31.421040 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4e5fc-518b-4004-a3e8-73b393732ce2 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:31.421328 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4e5fc-518b-4004-a3e8-73b393732ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:31.454009 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-9fe4e5fc-518b-4004-a3e8-73b393732ce2 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:33:31.455284 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-9fe4e5fc-518b-4004-a3e8-73b393732ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:31.493075 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:33:31.519925 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1458/2904] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:33:31.528065 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07854d04-69b4-4a69-b64f-64b72e2539c2 None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:31.528360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07854d04-69b4-4a69-b64f-64b72e2539c2 None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:31.528644 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07854d04-69b4-4a69-b64f-64b72e2539c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:31.549249 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-07854d04-69b4-4a69-b64f-64b72e2539c2 None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:33:31.550435 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-07854d04-69b4-4a69-b64f-64b72e2539c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:31.559987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1447/2905] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:33:31.565862 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4e5fc-518b-4004-a3e8-73b393732ce2 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:31.575960 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4e5fc-518b-4004-a3e8-73b393732ce2 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:31.580580 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fc2fd-8183-4ca1-a4ae-46de1fa5c9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:31.581246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fc2fd-8183-4ca1-a4ae-46de1fa5c9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:31.581525 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fc2fd-8183-4ca1-a4ae-46de1fa5c9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:31.581799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fc2fd-8183-4ca1-a4ae-46de1fa5c9b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:31.582551 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fc2fd-8183-4ca1-a4ae-46de1fa5c9b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86905) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:33:31.582860 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fc2fd-8183-4ca1-a4ae-46de1fa5c9b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:31.584447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fc2fd-8183-4ca1-a4ae-46de1fa5c9b5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:31.596026 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1459/2906] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Jul 25 19:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:33:31.600636 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07854d04-69b4-4a69-b64f-64b72e2539c2 None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:31.611543 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07854d04-69b4-4a69-b64f-64b72e2539c2 None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:31.616282 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19515b5f-a50b-4da9-9a30-7c2c521b3b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:31.616943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19515b5f-a50b-4da9-9a30-7c2c521b3b93 admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:31.617230 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19515b5f-a50b-4da9-9a30-7c2c521b3b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:31.617509 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19515b5f-a50b-4da9-9a30-7c2c521b3b93 admin admin] PATH_INFO: `/v3/projects` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:31.618932 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19515b5f-a50b-4da9-9a30-7c2c521b3b93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:31.620838 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19515b5f-a50b-4da9-9a30-7c2c521b3b93 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:31.720442 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1448/2907] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:33:31 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:33:31.725165 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fc2fd-8183-4ca1-a4ae-46de1fa5c9b5 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:31.736389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fc2fd-8183-4ca1-a4ae-46de1fa5c9b5 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:31.793570 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f20bc-b94c-403f-91b0-74ac37066a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:31.794241 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f20bc-b94c-403f-91b0-74ac37066a2c admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:31.794549 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f20bc-b94c-403f-91b0-74ac37066a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:31.794827 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f20bc-b94c-403f-91b0-74ac37066a2c admin admin] PATH_INFO: `/v3/users` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:31.795522 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f20bc-b94c-403f-91b0-74ac37066a2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:31.797435 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f20bc-b94c-403f-91b0-74ac37066a2c admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:31.896431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1460/2908] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:33:31 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:33:31.901567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19515b5f-a50b-4da9-9a30-7c2c521b3b93 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:31.911982 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19515b5f-a50b-4da9-9a30-7c2c521b3b93 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:31.917340 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a739bc-b4a0-4470-a892-47d75d0381e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:31.921299 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a739bc-b4a0-4470-a892-47d75d0381e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:31.921299 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a739bc-b4a0-4470-a892-47d75d0381e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:31.921299 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a739bc-b4a0-4470-a892-47d75d0381e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:31.921299 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a739bc-b4a0-4470-a892-47d75d0381e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:31.921299 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a739bc-b4a0-4470-a892-47d75d0381e0 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:31.931189 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1449/2909] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:33:31.935402 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f20bc-b94c-403f-91b0-74ac37066a2c admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:31.945870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f20bc-b94c-403f-91b0-74ac37066a2c admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:31.950816 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb45168-8214-44e4-98b7-fb4b57997181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:31.951527 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb45168-8214-44e4-98b7-fb4b57997181 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:31.951822 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb45168-8214-44e4-98b7-fb4b57997181 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:31.952107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb45168-8214-44e4-98b7-fb4b57997181 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:31.952879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb45168-8214-44e4-98b7-fb4b57997181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:31.954512 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb45168-8214-44e4-98b7-fb4b57997181 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:31.966211 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1461/2910] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:33:31.970540 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a739bc-b4a0-4470-a892-47d75d0381e0 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:31.981683 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a739bc-b4a0-4470-a892-47d75d0381e0 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:31.986477 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e07e02-0594-4e20-81fb-9a08b1a9fa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:31.987360 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e07e02-0594-4e20-81fb-9a08b1a9fa88 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:31.987647 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e07e02-0594-4e20-81fb-9a08b1a9fa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:31.987925 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e07e02-0594-4e20-81fb-9a08b1a9fa88 admin admin] PATH_INFO: `/v3/projects/03519fbb6e4c42098f809f799c77f884/users/31413326a6de469e9d657d9ef3037cbe/roles/9dc988eb162e47e48df810053eecb165` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:32.002943 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e07e02-0594-4e20-81fb-9a08b1a9fa88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03519fbb6e4c42098f809f799c77f884, user_id=31413326a6de469e9d657d9ef3037cbe, role_id=9dc988eb162e47e48df810053eecb165)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:32.005599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e07e02-0594-4e20-81fb-9a08b1a9fa88 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:32.079751 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1450/2911] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:33:31 2022] PUT /identity/v3/projects/03519fbb6e4c42098f809f799c77f884/users/31413326a6de469e9d657d9ef3037cbe/roles/9dc988eb162e47e48df810053eecb165 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:33:32.084926 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37874ece-05c3-45d7-ac20-f0e34d9f45ad None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:32.085217 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37874ece-05c3-45d7-ac20-f0e34d9f45ad None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:32.085499 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37874ece-05c3-45d7-ac20-f0e34d9f45ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:32.106029 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-37874ece-05c3-45d7-ac20-f0e34d9f45ad None None] MFA Rules not processed for user `4a0aec528900484294cd86eb90b642f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:33:32.107331 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-37874ece-05c3-45d7-ac20-f0e34d9f45ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:32.186560 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1462/2912] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:33:32.197017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e07e02-0594-4e20-81fb-9a08b1a9fa88 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:32.198779 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-97e07e02-0594-4e20-81fb-9a08b1a9fa88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:32.263928 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e07e02-0594-4e20-81fb-9a08b1a9fa88 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:32.267783 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756255cc-b043-4202-9987-89e9fa6a5583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:32.268439 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756255cc-b043-4202-9987-89e9fa6a5583 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:32.268731 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756255cc-b043-4202-9987-89e9fa6a5583 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:32.269175 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756255cc-b043-4202-9987-89e9fa6a5583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:32.279337 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756255cc-b043-4202-9987-89e9fa6a5583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:32.280772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756255cc-b043-4202-9987-89e9fa6a5583 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:32.291114 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1451/2913] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:33:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:33:33.139150 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37874ece-05c3-45d7-ac20-f0e34d9f45ad None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:33.140698 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-37874ece-05c3-45d7-ac20-f0e34d9f45ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:33.194908 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:33:33.220158 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37874ece-05c3-45d7-ac20-f0e34d9f45ad None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:33.226464 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199339dc-77bb-451b-9a1a-21c878d153bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:33.227198 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199339dc-77bb-451b-9a1a-21c878d153bc service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:33.227485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199339dc-77bb-451b-9a1a-21c878d153bc service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:33.227769 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199339dc-77bb-451b-9a1a-21c878d153bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:33.229542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-199339dc-77bb-451b-9a1a-21c878d153bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:33.241676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199339dc-77bb-451b-9a1a-21c878d153bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:33.243986 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199339dc-77bb-451b-9a1a-21c878d153bc service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:33.254145 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1463/2914] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:33:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:33:37.616775 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66965732-92aa-49dd-a470-5bdb5e01f00c None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:37.617396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66965732-92aa-49dd-a470-5bdb5e01f00c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:37.617396 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66965732-92aa-49dd-a470-5bdb5e01f00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:37.644044 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:33:37.655207 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-66965732-92aa-49dd-a470-5bdb5e01f00c None None] MFA Rules not processed for user `31413326a6de469e9d657d9ef3037cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:33:37.656448 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-66965732-92aa-49dd-a470-5bdb5e01f00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:37.731455 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1452/2915] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:33:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:33:37.737885 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199339dc-77bb-451b-9a1a-21c878d153bc service placement] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:37.751023 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199339dc-77bb-451b-9a1a-21c878d153bc service placement] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:37.759763 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaf76d-17d8-4f5c-833b-b2141bb55db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:37.760572 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf76d-17d8-4f5c-833b-b2141bb55db9 admin admin] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:37.760870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf76d-17d8-4f5c-833b-b2141bb55db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:37.761157 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf76d-17d8-4f5c-833b-b2141bb55db9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:37.761877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aaf76d-17d8-4f5c-833b-b2141bb55db9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:37.763828 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aaf76d-17d8-4f5c-833b-b2141bb55db9 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:37.852651 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1464/2916] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Jul 25 19:33:37 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:33:37.856876 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66965732-92aa-49dd-a470-5bdb5e01f00c None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:37.869074 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66965732-92aa-49dd-a470-5bdb5e01f00c None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:37.939338 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea0c12-ad2e-4bf4-92db-f8dfc344bddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:37.940031 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea0c12-ad2e-4bf4-92db-f8dfc344bddd admin admin] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:37.940320 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea0c12-ad2e-4bf4-92db-f8dfc344bddd admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:37.940625 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea0c12-ad2e-4bf4-92db-f8dfc344bddd admin admin] PATH_INFO: `/v3/users` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:37.941330 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea0c12-ad2e-4bf4-92db-f8dfc344bddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:37.943309 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea0c12-ad2e-4bf4-92db-f8dfc344bddd admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:38.045519 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1453/2917] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 19:33:37 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:33:38.049679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaf76d-17d8-4f5c-833b-b2141bb55db9 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:38.061079 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaf76d-17d8-4f5c-833b-b2141bb55db9 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:38.065940 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0d4c9-38c4-47b3-a340-b8be8f081e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:38.066679 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0d4c9-38c4-47b3-a340-b8be8f081e0d admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:38.066972 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0d4c9-38c4-47b3-a340-b8be8f081e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:38.067354 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0d4c9-38c4-47b3-a340-b8be8f081e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:38.068104 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0d4c9-38c4-47b3-a340-b8be8f081e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:38.069735 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0d4c9-38c4-47b3-a340-b8be8f081e0d admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:38.081213 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1465/2918] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:33:38.085567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea0c12-ad2e-4bf4-92db-f8dfc344bddd admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:38.096410 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea0c12-ad2e-4bf4-92db-f8dfc344bddd admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:38.101407 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4df2a8-5438-4bb5-b2ba-4e5246066dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:38.102934 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4df2a8-5438-4bb5-b2ba-4e5246066dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:38.103145 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4df2a8-5438-4bb5-b2ba-4e5246066dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:38.103423 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4df2a8-5438-4bb5-b2ba-4e5246066dc9 admin admin] PATH_INFO: `/v3/projects/39eb8ff4a4244a8dbb16e83ea3be4ce2/users/aa5b2379e1334d50914d6cddb033a56f/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:38.119601 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4df2a8-5438-4bb5-b2ba-4e5246066dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39eb8ff4a4244a8dbb16e83ea3be4ce2, user_id=aa5b2379e1334d50914d6cddb033a56f, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:38.122350 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4df2a8-5438-4bb5-b2ba-4e5246066dc9 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:38.217017 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1454/2919] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:33:38 2022] PUT /identity/v3/projects/39eb8ff4a4244a8dbb16e83ea3be4ce2/users/aa5b2379e1334d50914d6cddb033a56f/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:33:38.221277 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0d4c9-38c4-47b3-a340-b8be8f081e0d admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:38.232577 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0d4c9-38c4-47b3-a340-b8be8f081e0d admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:38.288878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336b5f8-fd6d-4219-9a5f-74d2e4cf9f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:38.290120 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336b5f8-fd6d-4219-9a5f-74d2e4cf9f90 admin admin] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:38.290447 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336b5f8-fd6d-4219-9a5f-74d2e4cf9f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:38.290733 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336b5f8-fd6d-4219-9a5f-74d2e4cf9f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:38.291463 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336b5f8-fd6d-4219-9a5f-74d2e4cf9f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:38.293081 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336b5f8-fd6d-4219-9a5f-74d2e4cf9f90 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:38.303819 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1466/2920] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Jul 25 19:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:33:38.308869 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4df2a8-5438-4bb5-b2ba-4e5246066dc9 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:38.320134 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4df2a8-5438-4bb5-b2ba-4e5246066dc9 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:38.325403 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3882e88-445d-4f23-86d9-84f005ff83e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:38.326319 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3882e88-445d-4f23-86d9-84f005ff83e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:38.326639 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3882e88-445d-4f23-86d9-84f005ff83e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:38.326923 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3882e88-445d-4f23-86d9-84f005ff83e0 admin admin] PATH_INFO: `/v3/projects/39eb8ff4a4244a8dbb16e83ea3be4ce2/users/aa5b2379e1334d50914d6cddb033a56f/roles/9bfe07ad221449c586c8e262f86d7f48` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:38.329084 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3882e88-445d-4f23-86d9-84f005ff83e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39eb8ff4a4244a8dbb16e83ea3be4ce2, user_id=aa5b2379e1334d50914d6cddb033a56f, role_id=9bfe07ad221449c586c8e262f86d7f48)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:38.331821 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3882e88-445d-4f23-86d9-84f005ff83e0 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:38.345263 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1455/2921] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Jul 25 19:33:38 2022] PUT /identity/v3/projects/39eb8ff4a4244a8dbb16e83ea3be4ce2/users/aa5b2379e1334d50914d6cddb033a56f/roles/9bfe07ad221449c586c8e262f86d7f48 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:33:44.037183 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06f414-7140-4ced-aae1-6eb11cbd213a None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:44.037731 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06f414-7140-4ced-aae1-6eb11cbd213a None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:44.037842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06f414-7140-4ced-aae1-6eb11cbd213a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:44.070306 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-ce06f414-7140-4ced-aae1-6eb11cbd213a None None] MFA Rules not processed for user `aa5b2379e1334d50914d6cddb033a56f`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:33:44.073009 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-ce06f414-7140-4ced-aae1-6eb11cbd213a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:44.157135 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1467/2922] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:33:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:33:44.165924 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3882e88-445d-4f23-86d9-84f005ff83e0 admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:44.168201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-a3882e88-445d-4f23-86d9-84f005ff83e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:44.225023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:33:44.248087 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3882e88-445d-4f23-86d9-84f005ff83e0 admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:44.253907 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e8952-ae9a-48c1-b86c-a8e77276e3c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:44.254592 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e8952-ae9a-48c1-b86c-a8e77276e3c3 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:44.254882 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e8952-ae9a-48c1-b86c-a8e77276e3c3 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:44.255165 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e8952-ae9a-48c1-b86c-a8e77276e3c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:44.316306 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e8952-ae9a-48c1-b86c-a8e77276e3c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:44.317989 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e8952-ae9a-48c1-b86c-a8e77276e3c3 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:44.329000 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1456/2923] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:33:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:33:54.146336 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=89,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:33:54.335557 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=33 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:34:09.939680 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a310939-0b21-4ba7-bc83-78b19f31d123 None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:34:09.940280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a310939-0b21-4ba7-bc83-78b19f31d123 None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:34:09.940280 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a310939-0b21-4ba7-bc83-78b19f31d123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:34:09.960423 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-1a310939-0b21-4ba7-bc83-78b19f31d123 None None] MFA Rules not processed for user `31413326a6de469e9d657d9ef3037cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:34:09.961633 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1a310939-0b21-4ba7-bc83-78b19f31d123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:34:09.968601 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1468/2924] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:34:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:34:10.306167 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e8952-ae9a-48c1-b86c-a8e77276e3c3 service nova] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:34:10.307614 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b66e8952-ae9a-48c1-b86c-a8e77276e3c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:34:10.367892 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e8952-ae9a-48c1-b86c-a8e77276e3c3 service nova] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:34:10.371616 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302b481-56e2-4bce-bbf7-96edf713d39a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:34:10.372303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302b481-56e2-4bce-bbf7-96edf713d39a service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:34:10.372597 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302b481-56e2-4bce-bbf7-96edf713d39a service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:34:10.372877 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302b481-56e2-4bce-bbf7-96edf713d39a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:34:10.374824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9302b481-56e2-4bce-bbf7-96edf713d39a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:34:10.386553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302b481-56e2-4bce-bbf7-96edf713d39a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:34:10.388016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302b481-56e2-4bce-bbf7-96edf713d39a service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:34:10.398209 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1457/2925] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:34:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:34:13.074409 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a310939-0b21-4ba7-bc83-78b19f31d123 None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:34:13.075937 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-1a310939-0b21-4ba7-bc83-78b19f31d123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:34:13.150914 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a310939-0b21-4ba7-bc83-78b19f31d123 None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:34:13.161167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6fbcc-9acc-4266-8256-f5f1b540fc86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:34:13.161328 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6fbcc-9acc-4266-8256-f5f1b540fc86 service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:34:13.161542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6fbcc-9acc-4266-8256-f5f1b540fc86 service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:34:13.161799 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6fbcc-9acc-4266-8256-f5f1b540fc86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:34:13.164565 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-28c6fbcc-9acc-4266-8256-f5f1b540fc86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:34:13.178398 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6fbcc-9acc-4266-8256-f5f1b540fc86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:34:13.180010 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6fbcc-9acc-4266-8256-f5f1b540fc86 service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:34:13.191532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1469/2926] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:34:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:34:20.393352 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=26 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:34:23.186251 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=35 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:34:32.184504 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302b481-56e2-4bce-bbf7-96edf713d39a service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:34:32.186047 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9302b481-56e2-4bce-bbf7-96edf713d39a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:34:32.236062 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302b481-56e2-4bce-bbf7-96edf713d39a service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:34:32.242299 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17345eb-0883-4ab3-a6b2-4f5e1cf6ce33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:34:32.242985 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17345eb-0883-4ab3-a6b2-4f5e1cf6ce33 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:34:32.243277 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17345eb-0883-4ab3-a6b2-4f5e1cf6ce33 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:34:32.243553 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17345eb-0883-4ab3-a6b2-4f5e1cf6ce33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:34:32.246246 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-b17345eb-0883-4ab3-a6b2-4f5e1cf6ce33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:34:32.262304 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17345eb-0883-4ab3-a6b2-4f5e1cf6ce33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:34:32.264484 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17345eb-0883-4ab3-a6b2-4f5e1cf6ce33 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:34:32.280854 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1458/2927] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:34:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:34:35.659443 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6fbcc-9acc-4266-8256-f5f1b540fc86 service placement] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:34:35.669181 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6fbcc-9acc-4266-8256-f5f1b540fc86 service placement] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:34:35.672815 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8c4d9-d48e-4d1d-bf78-d57df9e3b3cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:34:35.673459 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8c4d9-d48e-4d1d-bf78-d57df9e3b3cb service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:34:35.673740 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8c4d9-d48e-4d1d-bf78-d57df9e3b3cb service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:34:35.674017 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8c4d9-d48e-4d1d-bf78-d57df9e3b3cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:34:35.684011 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8c4d9-d48e-4d1d-bf78-d57df9e3b3cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:34:35.685429 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8c4d9-d48e-4d1d-bf78-d57df9e3b3cb service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:34:35.695840 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1470/2928] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:34:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:34:42.274535 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=18 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:34:45.690761 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=6 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:35:27.906752 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17345eb-0883-4ab3-a6b2-4f5e1cf6ce33 service nova] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:35:27.916755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17345eb-0883-4ab3-a6b2-4f5e1cf6ce33 service nova] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:35:27.920549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c6c88-6326-4a3b-804c-7da393859988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:35:27.921242 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c6c88-6326-4a3b-804c-7da393859988 service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:35:27.921648 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c6c88-6326-4a3b-804c-7da393859988 service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:35:27.921940 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c6c88-6326-4a3b-804c-7da393859988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:35:27.923762 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-085c6c88-6326-4a3b-804c-7da393859988 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:35:27.935033 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c6c88-6326-4a3b-804c-7da393859988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:35:27.936465 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c6c88-6326-4a3b-804c-7da393859988 service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:35:27.945356 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1459/2929] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:35:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:35:37.941303 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=6 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:36:54.769897 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8c4d9-d48e-4d1d-bf78-d57df9e3b3cb service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:36:54.780138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8c4d9-d48e-4d1d-bf78-d57df9e3b3cb service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:36:54.784867 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3ab43-1c3f-457d-b5be-97a7c457bf7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:36:54.785532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3ab43-1c3f-457d-b5be-97a7c457bf7f service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:36:54.785822 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3ab43-1c3f-457d-b5be-97a7c457bf7f service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:36:54.786167 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3ab43-1c3f-457d-b5be-97a7c457bf7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:36:54.787994 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-64a3ab43-1c3f-457d-b5be-97a7c457bf7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:36:54.803957 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3ab43-1c3f-457d-b5be-97a7c457bf7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:36:54.805390 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3ab43-1c3f-457d-b5be-97a7c457bf7f service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:36:54.816950 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1471/2930] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:36:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:36:58.234014 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315dc55-d3de-4425-bc7f-03fed348e26c None None] REQUEST_METHOD: `POST` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:36:58.234578 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315dc55-d3de-4425-bc7f-03fed348e26c None None] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:36:58.234688 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315dc55-d3de-4425-bc7f-03fed348e26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:36:58.254528 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.auth.core [None req-d315dc55-d3de-4425-bc7f-03fed348e26c None None] MFA Rules not processed for user `31413326a6de469e9d657d9ef3037cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=86906) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:36:58.255773 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-d315dc55-d3de-4425-bc7f-03fed348e26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:36:58.263225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1460/2931] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:36:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:36:58.592038 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3ab43-1c3f-457d-b5be-97a7c457bf7f service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:36:58.601176 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3ab43-1c3f-457d-b5be-97a7c457bf7f service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:36:58.605532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475bb4d-46a9-4289-99e4-c71928a88d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:36:58.606162 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475bb4d-46a9-4289-99e4-c71928a88d3e service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:36:58.606485 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475bb4d-46a9-4289-99e4-c71928a88d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:36:58.606766 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475bb4d-46a9-4289-99e4-c71928a88d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:36:58.608504 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-c475bb4d-46a9-4289-99e4-c71928a88d3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:36:58.617959 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475bb4d-46a9-4289-99e4-c71928a88d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:36:58.619431 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475bb4d-46a9-4289-99e4-c71928a88d3e service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:36:58.627586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1472/2932] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:36:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:37:08.249326 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:37:08.623838 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=12 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:37:20.378172 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315dc55-d3de-4425-bc7f-03fed348e26c None None] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:37:20.393046 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315dc55-d3de-4425-bc7f-03fed348e26c None None] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:37:20.398981 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b9518-de4f-4cb0-9f06-c4e45d66bdf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:37:20.399863 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9518-de4f-4cb0-9f06-c4e45d66bdf5 service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:37:20.400224 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9518-de4f-4cb0-9f06-c4e45d66bdf5 service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:37:20.400576 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9518-de4f-4cb0-9f06-c4e45d66bdf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:37:20.403182 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9d4b9518-de4f-4cb0-9f06-c4e45d66bdf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:37:20.417215 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b9518-de4f-4cb0-9f06-c4e45d66bdf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:37:20.419471 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b9518-de4f-4cb0-9f06-c4e45d66bdf5 service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:37:20.431676 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1461/2933] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:37:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:37:30.427440 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=6 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:38:39.143193 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475bb4d-46a9-4289-99e4-c71928a88d3e service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:38:39.156547 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475bb4d-46a9-4289-99e4-c71928a88d3e service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:38:39.164586 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ddbd3f-f977-4e80-bff6-8688f0714e3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:38:39.165265 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddbd3f-f977-4e80-bff6-8688f0714e3c service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:38:39.165542 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddbd3f-f977-4e80-bff6-8688f0714e3c service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:38:39.165830 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddbd3f-f977-4e80-bff6-8688f0714e3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:38:39.175915 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ddbd3f-f977-4e80-bff6-8688f0714e3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:38:39.177521 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ddbd3f-f977-4e80-bff6-8688f0714e3c service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:38:39.188506 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1473/2934] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:38:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:38:49.184005 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=6 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:39:10.572824 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b9518-de4f-4cb0-9f06-c4e45d66bdf5 service nova] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:39:10.574225 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-9d4b9518-de4f-4cb0-9f06-c4e45d66bdf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:39:10.653930 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b9518-de4f-4cb0-9f06-c4e45d66bdf5 service nova] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:39:10.657739 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb436458-3544-40e1-b77e-621bd4f2b06a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:39:10.658424 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb436458-3544-40e1-b77e-621bd4f2b06a service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:39:10.658720 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb436458-3544-40e1-b77e-621bd4f2b06a service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:39:10.659005 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb436458-3544-40e1-b77e-621bd4f2b06a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:39:10.667777 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb436458-3544-40e1-b77e-621bd4f2b06a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:39:10.669250 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb436458-3544-40e1-b77e-621bd4f2b06a service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:39:10.679091 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1462/2935] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:39:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:39:20.675130 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=35 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:39:23.276035 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d35b1-9ed1-4f5a-8470-db911f22d8ed None None] REQUEST_METHOD: `POST` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:39:23.276581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d35b1-9ed1-4f5a-8470-db911f22d8ed None None] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:39:23.276581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d35b1-9ed1-4f5a-8470-db911f22d8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:39:23.296412 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.auth.core [None req-162d35b1-9ed1-4f5a-8470-db911f22d8ed None None] MFA Rules not processed for user `31413326a6de469e9d657d9ef3037cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=86905) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:39:23.297644 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-162d35b1-9ed1-4f5a-8470-db911f22d8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:39:23.305469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1474/2936] 10.222.0.17 () {58 vars in 958 bytes} [Mon Jul 25 19:39:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:39:23.344305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb436458-3544-40e1-b77e-621bd4f2b06a service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:39:23.346479 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-fb436458-3544-40e1-b77e-621bd4f2b06a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:39:23.358968 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb436458-3544-40e1-b77e-621bd4f2b06a service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:39:23.362836 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559eca24-f3d9-49f0-ae9a-7ad15848734e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a419d02a7dc46e9b3c5b3d4eb3151d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:39:23.363483 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559eca24-f3d9-49f0-ae9a-7ad15848734e service cinder] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:39:23.363770 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559eca24-f3d9-49f0-ae9a-7ad15848734e service cinder] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:39:23.364051 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559eca24-f3d9-49f0-ae9a-7ad15848734e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:39:23.365843 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-559eca24-f3d9-49f0-ae9a-7ad15848734e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:39:23.379861 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559eca24-f3d9-49f0-ae9a-7ad15848734e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:39:23.381300 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559eca24-f3d9-49f0-ae9a-7ad15848734e service cinder] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:39:23.393549 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1463/2937] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:39:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:39:26.573400 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d35b1-9ed1-4f5a-8470-db911f22d8ed None None] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:39:26.585982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d35b1-9ed1-4f5a-8470-db911f22d8ed None None] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:39:26.591007 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13717eae-e5f6-476e-9965-b2149d05e917 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:39:26.591955 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13717eae-e5f6-476e-9965-b2149d05e917 service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:39:26.592246 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13717eae-e5f6-476e-9965-b2149d05e917 service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:39:26.592529 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13717eae-e5f6-476e-9965-b2149d05e917 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:39:26.601751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13717eae-e5f6-476e-9965-b2149d05e917 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:39:26.603397 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13717eae-e5f6-476e-9965-b2149d05e917 service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:39:26.613568 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1475/2938] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:39:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:39:33.262253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559eca24-f3d9-49f0-ae9a-7ad15848734e service cinder] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:39:33.276967 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559eca24-f3d9-49f0-ae9a-7ad15848734e service cinder] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:39:33.283591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca4457-7152-47a8-a34c-4f17dc2d11cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:39:33.284238 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca4457-7152-47a8-a34c-4f17dc2d11cd service nova] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:39:33.284579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca4457-7152-47a8-a34c-4f17dc2d11cd service nova] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:39:33.284796 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca4457-7152-47a8-a34c-4f17dc2d11cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:39:33.286772 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-c6ca4457-7152-47a8-a34c-4f17dc2d11cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:39:33.300392 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca4457-7152-47a8-a34c-4f17dc2d11cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:39:33.302188 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca4457-7152-47a8-a34c-4f17dc2d11cd service nova] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:39:33.318195 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1464/2939] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:39:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:39:36.609332 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=9 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:39:43.310201 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=12 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:39:45.666645 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13717eae-e5f6-476e-9965-b2149d05e917 service placement] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:39:45.680610 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13717eae-e5f6-476e-9965-b2149d05e917 service placement] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:39:45.686134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458dc51-37bc-4e76-96f7-214768d1ae48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e628cb75e0c74f27a390945c8a04335c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:39:45.686822 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458dc51-37bc-4e76-96f7-214768d1ae48 service nova] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:39:45.687107 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458dc51-37bc-4e76-96f7-214768d1ae48 service nova] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:39:45.687389 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458dc51-37bc-4e76-96f7-214768d1ae48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:39:45.697227 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458dc51-37bc-4e76-96f7-214768d1ae48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:39:45.698842 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458dc51-37bc-4e76-96f7-214768d1ae48 service nova] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:39:45.710388 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1476/2940] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:39:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:39:49.175031 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca4457-7152-47a8-a34c-4f17dc2d11cd service nova] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:39:49.185591 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca4457-7152-47a8-a34c-4f17dc2d11cd service nova] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:39:49.193284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff1213-b39e-4e35-8edb-48a9683b2a4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:39:49.193284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff1213-b39e-4e35-8edb-48a9683b2a4a service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:39:49.193284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff1213-b39e-4e35-8edb-48a9683b2a4a service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:39:49.193284 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff1213-b39e-4e35-8edb-48a9683b2a4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:39:49.200900 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff1213-b39e-4e35-8edb-48a9683b2a4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:39:49.202474 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff1213-b39e-4e35-8edb-48a9683b2a4a service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:39:49.213417 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1465/2941] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:39:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:39:55.705115 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=6 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:39:59.208829 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=6 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:40:31.918139 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458dc51-37bc-4e76-96f7-214768d1ae48 service nova] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:40:31.947676 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458dc51-37bc-4e76-96f7-214768d1ae48 service nova] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:40:31.951419 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765819d4-1853-4731-b8cc-2fa89a420f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:40:31.956532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765819d4-1853-4731-b8cc-2fa89a420f9c service neutron] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:40:31.956532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765819d4-1853-4731-b8cc-2fa89a420f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:40:31.956532 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765819d4-1853-4731-b8cc-2fa89a420f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:40:31.960454 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765819d4-1853-4731-b8cc-2fa89a420f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:40:31.961877 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765819d4-1853-4731-b8cc-2fa89a420f9c service neutron] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:40:31.971376 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1477/2942] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:40:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:40:41.966643 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=13 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:41:55.708831 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff1213-b39e-4e35-8edb-48a9683b2a4a service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:41:55.745903 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff1213-b39e-4e35-8edb-48a9683b2a4a service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:41:55.749604 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d865f3-9a89-4a80-af84-7a93b0d93afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffcf280bc6d54b2fb2d22634468ac6cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:41:55.750253 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d865f3-9a89-4a80-af84-7a93b0d93afd service neutron] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:41:55.750579 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d865f3-9a89-4a80-af84-7a93b0d93afd service neutron] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:41:55.750868 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d865f3-9a89-4a80-af84-7a93b0d93afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:41:55.837491 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d865f3-9a89-4a80-af84-7a93b0d93afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:41:55.839406 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d865f3-9a89-4a80-af84-7a93b0d93afd service neutron] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:41:55.849099 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1466/2943] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:41:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:41:59.540127 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765819d4-1853-4731-b8cc-2fa89a420f9c service neutron] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:41:59.567879 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765819d4-1853-4731-b8cc-2fa89a420f9c service neutron] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:41:59.573112 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bc225-bec9-4e6c-a0a0-4ac3e00ec988 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:41:59.573766 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bc225-bec9-4e6c-a0a0-4ac3e00ec988 service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:41:59.574050 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bc225-bec9-4e6c-a0a0-4ac3e00ec988 service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:41:59.574341 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bc225-bec9-4e6c-a0a0-4ac3e00ec988 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:41:59.576130 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-e62bc225-bec9-4e6c-a0a0-4ac3e00ec988 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:41:59.602884 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bc225-bec9-4e6c-a0a0-4ac3e00ec988 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:41:59.605338 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bc225-bec9-4e6c-a0a0-4ac3e00ec988 service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:41:59.615891 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1478/2944] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:41:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:42:03.225713 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d865f3-9a89-4a80-af84-7a93b0d93afd service neutron] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:42:03.238023 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d865f3-9a89-4a80-af84-7a93b0d93afd service neutron] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:42:03.243203 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e6f57-2117-41de-9f66-dafb35afdfaf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:42:03.243894 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e6f57-2117-41de-9f66-dafb35afdfaf service placement] REQUEST_METHOD: `GET` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:42:03.244186 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e6f57-2117-41de-9f66-dafb35afdfaf service placement] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:42:03.244711 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e6f57-2117-41de-9f66-dafb35afdfaf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:42:03.246617 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-816e6f57-2117-41de-9f66-dafb35afdfaf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:42:03.259987 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e6f57-2117-41de-9f66-dafb35afdfaf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:42:03.261808 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e6f57-2117-41de-9f66-dafb35afdfaf service placement] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:42:03.272811 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1467/2945] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:42:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:42:03.350804 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bc225-bec9-4e6c-a0a0-4ac3e00ec988 service placement] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:42:03.363116 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bc225-bec9-4e6c-a0a0-4ac3e00ec988 service placement] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:42:03.367646 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:42:03.368638 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:42:03.369486 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:42:03.369826 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] PATH_INFO: `/v3/users/31413326a6de469e9d657d9ef3037cbe` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:42:03.371062 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] RBAC: Authorizing `identity:delete_user(user_id=31413326a6de469e9d657d9ef3037cbe)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:42:03.372982 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:42:03.569138 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31413326a6de469e9d657d9ef3037cbe'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:03.569748 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31413326a6de469e9d657d9ef3037cbe'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:03.577845 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31413326a6de469e9d657d9ef3037cbe'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:03.590899 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31413326a6de469e9d657d9ef3037cbe'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:03.647911 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31413326a6de469e9d657d9ef3037cbe'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:03.655232 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1479/2946] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:42:03 2022] DELETE /identity/v3/users/31413326a6de469e9d657d9ef3037cbe => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:42:03.907755 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e6f57-2117-41de-9f66-dafb35afdfaf service placement] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:42:03.923571 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e6f57-2117-41de-9f66-dafb35afdfaf service placement] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:42:03.990599 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:42:03.991478 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:42:03.991770 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:42:03.992084 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] PATH_INFO: `/v3/projects/03519fbb6e4c42098f809f799c77f884` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:42:03.993305 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] RBAC: Authorizing `identity:delete_project(project_id=03519fbb6e4c42098f809f799c77f884)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:42:03.996264 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:42:04.114598 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:42:04.118522 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03519fbb6e4c42098f809f799c77f884', 'request_id': 'req-33ed897a-1e29-441d-8816-f48a68cc09de'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:04.190556 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03519fbb6e4c42098f809f799c77f884', 'request_id': 'req-33ed897a-1e29-441d-8816-f48a68cc09de'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:04.196765 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03519fbb6e4c42098f809f799c77f884', 'request_id': 'req-33ed897a-1e29-441d-8816-f48a68cc09de'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:04.208506 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] The token cache is being invalidate because project 03519fbb6e4c42098f809f799c77f884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:42:04.208806 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:04.209469 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:04.211048 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1468/2947] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:42:03 2022] DELETE /identity/v3/projects/03519fbb6e4c42098f809f799c77f884 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:42:04.216034 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:42:04.217469 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:42:04.231606 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6c4ce-383c-49eb-914f-f498ced9b27b admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:42:04.294692 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:42:04.295579 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:42:04.295870 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:42:04.296153 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] PATH_INFO: `/v3/users/aa5b2379e1334d50914d6cddb033a56f` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:42:04.297373 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5b2379e1334d50914d6cddb033a56f)` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:42:04.299439 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:42:04.477180 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:42:04.482875 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5b2379e1334d50914d6cddb033a56f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:04.483581 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5b2379e1334d50914d6cddb033a56f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:04.490690 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5b2379e1334d50914d6cddb033a56f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:04.503066 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5b2379e1334d50914d6cddb033a56f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:04.570979 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.notifications [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5b2379e1334d50914d6cddb033a56f'} {{(pid=86905) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:04.577999 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1480/2948] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Jul 25 19:42:04 2022] DELETE /identity/v3/users/aa5b2379e1334d50914d6cddb033a56f => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:42:04.830287 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] Authenticating user token {{(pid=86906) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:42:04.831862 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.fernet_utils [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86906) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:42:04.846545 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed897a-1e29-441d-8816-f48a68cc09de admin admin] Validating token access rules against request {{(pid=86906) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:42:04.907567 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca76c8-665b-4c7f-b503-fe0004cbd974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a0aec528900484294cd86eb90b642f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c39adca8a83c49e4a2ab45e68b3a9724', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86906) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:42:04.908438 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca76c8-665b-4c7f-b503-fe0004cbd974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:42:04.908729 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca76c8-665b-4c7f-b503-fe0004cbd974 admin admin] SCRIPT_NAME: `/identity` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:42:04.909016 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca76c8-665b-4c7f-b503-fe0004cbd974 admin admin] PATH_INFO: `/v3/projects/39eb8ff4a4244a8dbb16e83ea3be4ce2` {{(pid=86906) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:42:04.910210 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca76c8-665b-4c7f-b503-fe0004cbd974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39eb8ff4a4244a8dbb16e83ea3be4ce2)` {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:42:04.912173 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca76c8-665b-4c7f-b503-fe0004cbd974 admin admin] RBAC: Authorization granted {{(pid=86906) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:42:05.038347 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7eca76c8-665b-4c7f-b503-fe0004cbd974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39eb8ff4a4244a8dbb16e83ea3be4ce2', 'request_id': 'req-7eca76c8-665b-4c7f-b503-fe0004cbd974'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:05.102191 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7eca76c8-665b-4c7f-b503-fe0004cbd974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39eb8ff4a4244a8dbb16e83ea3be4ce2', 'request_id': 'req-7eca76c8-665b-4c7f-b503-fe0004cbd974'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:05.109585 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7eca76c8-665b-4c7f-b503-fe0004cbd974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39eb8ff4a4244a8dbb16e83ea3be4ce2', 'request_id': 'req-7eca76c8-665b-4c7f-b503-fe0004cbd974'} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:05.121329 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7eca76c8-665b-4c7f-b503-fe0004cbd974 admin admin] The token cache is being invalidate because project 39eb8ff4a4244a8dbb16e83ea3be4ce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86906) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:42:05.121627 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7eca76c8-665b-4c7f-b503-fe0004cbd974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:05.122291 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG keystone.notifications [None req-7eca76c8-665b-4c7f-b503-fe0004cbd974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86906) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:42:05.123931 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: [pid: 86906|app: 0|req: 1469/2949] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Jul 25 19:42:04 2022] DELETE /identity/v3/projects/39eb8ff4a4244a8dbb16e83ea3be4ce2 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:42:14.647895 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=14,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:42:15.118077 dv-cinder-820027-16-iscsi devstack@keystone.service[86906]: DEBUG dbcounter [-] [86906] Writing DB stats keystone:SELECT=59,keystone:DELETE=11,keystone:INSERT=2 {{(pid=86906) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:43:42.130059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] Authenticating user token {{(pid=86905) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:43:42.130059 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:43:42.225751 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f56984-137c-49b5-890a-5a0a00386aa5 admin admin] Validating token access rules against request {{(pid=86905) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:43:42.286134 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec288c-cc43-4a0f-a4bf-39a382b7d10d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5027a60da214afda9144b5ae6cce15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d7c3b139f284435aea4ff08a2f0952b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86905) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:43:42.286832 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec288c-cc43-4a0f-a4bf-39a382b7d10d service placement] REQUEST_METHOD: `GET` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:43:42.287123 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec288c-cc43-4a0f-a4bf-39a382b7d10d service placement] SCRIPT_NAME: `/identity` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:43:42.287406 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec288c-cc43-4a0f-a4bf-39a382b7d10d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86905) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:43:42.289224 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.fernet_utils [None req-eaec288c-cc43-4a0f-a4bf-39a382b7d10d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86905) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:43:42.301399 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec288c-cc43-4a0f-a4bf-39a382b7d10d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:43:42.303097 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec288c-cc43-4a0f-a4bf-39a382b7d10d service placement] RBAC: Authorization granted {{(pid=86905) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:43:42.314111 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: [pid: 86905|app: 0|req: 1481/2950] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 19:43:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:43:52.308878 dv-cinder-820027-16-iscsi devstack@keystone.service[86905]: DEBUG dbcounter [-] [86905] Writing DB stats keystone:SELECT=67 {{(pid=86905) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}